Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [...........................XXX..............................] duration: 465.82392ms FAIL (2025-07-08 05:24:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 8) ======LOGS====== pload=284991]: Uploading piece... Jul 8 05:00:37.137063656 (355h49m4.063358349s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.137984339 (355h49m4.064279032s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138034396 (355h49m4.064329092s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138836689 (355h49m4.065131384s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138884136 (355h49m4.065178829s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139428483 (355h49m4.065723178s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139436408 (355h49m4.0657311s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139788656 (355h49m4.066083348s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139920627 (355h49m4.066215325s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140071882 (355h49m4.066366572s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140353654 (355h49m4.06664834s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140415319 (355h49m4.066710015s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140681286 (355h49m4.066975982s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140973150 (355h49m4.067267847s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.144728804 (355h49m4.071023482s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.249883590 (355h49m4.176178265s) [upload=284991]: Next returned nil segment Jul 8 05:00:37.249902715 (355h49m4.176197412s) [upload=284991]: Waiting for error group managing segments... Jul 8 05:00:37.260212837 (355h49m4.186507526s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260803813 (355h49m4.187098502s) [noise=true , address=104.36.82.45:4915 , piece_id=G6IVO3DOQGN5GQKW67NX2ZKBOVTQEBA2PUNAGFDIKUI6OLPGMZHQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260834568 (355h49m4.187129264s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261367510 (355h49m4.187662188s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261455746 (355h49m4.187750422s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261903476 (355h49m4.188198155s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.278055422 (355h49m4.204350095s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.300910056 (355h49m4.227204736s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:01:37.042774308 (355h50m3.969069003s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:19LE2Lq4e6HojnMp9xQ3ySLkRe5PemXTwiQZThN5jT46bQhN6K, address:77.90.31.19:4855): protocol: rpc: dial tcp 77.90.31.19:4855: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042887419 (355h50m3.969182106s) [noise=true , address=77.90.31.16:4747 , piece_id=HV6IOGCMOHIUEYTGJMQ62V4UUNJ2MQD37SFF5Y23X2S66YYQSN3Q, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12ERdBm3XPw4s7JHeMWsVniwcqnfMtP4ZX8sBVjri6dzZtq8AF2, address:77.90.31.16:4747): protocol: rpc: dial tcp 77.90.31.16:4747: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042966233 (355h50m3.969260923s) [noise=true , address=77.90.31.19:4856 , piece_id=CR7AX6HZZGBQM5QATGLDLHQGUPZC4KW6H5CIBB4LFRTNL2WOLQFQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:1MntjJtcCDSRozKpesp6S2ZjYrMhxiQheDiPMPSCnavWBBGYt3, address:77.90.31.19:4856): protocol: rpc: dial tcp 77.90.31.19:4856: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043090813 (355h50m3.969385503s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: write tcp 159.223.143.39:47434->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043180860 (355h50m3.969475548s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12Ttdx6LHmuAt8JVC7arCDgWjm9eLadjeaEw17F12btoXuSiq9t, address:77.90.31.11:4583): protocol: write tcp 159.223.143.39:43298->77.90.31.11:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043193162 (355h50m3.969487876s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12LuFBTJg1aNmZoEdiQTtvfnfh5qnWVZFtqFX93L4fNfiYTYqJ6, address:77.90.31.18:4818): protocol: rpc: dial tcp 77.90.31.18:4818: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043283221 (355h50m3.969577895s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:126Sou5pnMm2eE2FJpv6WYX2VSqJvZZf9rFFJD6hFYDetRD28zu, address:77.90.31.12:4603): protocol: rpc: dial tcp 77.90.31.12:4603: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043307928 (355h50m3.969602654s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043494443 (355h50m3.969789132s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:168Tw7W69Tq9opDKhJ4cM8PGnzpxp9E9SxKbdVxJeF83pETqGx, address:77.90.31.17:4804): protocol: rpc: dial tcp 77.90.31.17:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043560259 (355h50m3.969854956s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: write tcp 159.223.143.39:37628->77.90.31.13:4632: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043598718 (355h50m3.969893411s) [seg_pos={0 0}, upload=284991]: Done waiting for segment. successful: 8 optimal: 14 errs: <nil> Jul 8 05:01:37.086050110 (355h50m4.012344788s) [upload=284991]: Done uploading segments... Jul 8 05:01:37.086091388 (355h50m4.012386065s) [upload=284991]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 8) Duration: 465.853664ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................XXXX..................................] duration: 460.746908ms FAIL (2025-07-08 05:24:15 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== GPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674108143 (355h49m20.600402838s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674596780 (355h49m20.600891471s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675604361 (355h49m20.601899058s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675768580 (355h49m20.602063277s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676120119 (355h49m20.60241481s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676215473 (355h49m20.602510169s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676623023 (355h49m20.602917716s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676647235 (355h49m20.602941929s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676696624 (355h49m20.602991321s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677177615 (355h49m20.603472312s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677707610 (355h49m20.604002306s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678250080 (355h49m20.604544775s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678907662 (355h49m20.605202358s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679453605 (355h49m20.6057483s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679618853 (355h49m20.605913551s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.771112310 (355h49m20.697407s) [upload=284997]: Next returned nil segment Jul 8 05:00:53.771144706 (355h49m20.697439396s) [upload=284997]: Waiting for error group managing segments... Jul 8 05:00:53.780217385 (355h49m20.70651207s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.780770161 (355h49m20.70706484s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781213801 (355h49m20.707508488s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781712476 (355h49m20.708007155s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.782192975 (355h49m20.708487647s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:01:23.612230509 (355h49m50.538525197s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612301272 (355h49m50.538595951s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 159.223.143.39:45850->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612349516 (355h49m50.538644191s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 159.223.143.39:38418->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612454902 (355h49m50.538749592s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612531645 (355h49m50.538826322s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 159.223.143.39:39432->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612552075 (355h49m50.538846753s) [noise=true , address=45.91.116.11:4118 , piece_id=KFPPGPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 159.223.143.39:33394->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612586657 (355h49m50.538881334s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 159.223.143.39:49070->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612604791 (355h49m50.538899468s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612642895 (355h49m50.538937572s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612651542 (355h49m50.538946218s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 159.223.143.39:47692->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612680902 (355h49m50.538975589s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612713168 (355h49m50.539007856s) [seg_pos={0 0}, upload=284997]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 8 05:01:23.648975097 (355h49m50.575269777s) [upload=284997]: Done uploading segments... Jul 8 05:01:23.649010451 (355h49m50.575305138s) [upload=284997]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 460.779736ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................XXXX.................................] duration: 443.609954ms FAIL (2025-07-08 05:23:54 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== GPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674108143 (355h49m20.600402838s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674596780 (355h49m20.600891471s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675604361 (355h49m20.601899058s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675768580 (355h49m20.602063277s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676120119 (355h49m20.60241481s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676215473 (355h49m20.602510169s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676623023 (355h49m20.602917716s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676647235 (355h49m20.602941929s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676696624 (355h49m20.602991321s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677177615 (355h49m20.603472312s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677707610 (355h49m20.604002306s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678250080 (355h49m20.604544775s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678907662 (355h49m20.605202358s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679453605 (355h49m20.6057483s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679618853 (355h49m20.605913551s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.771112310 (355h49m20.697407s) [upload=284997]: Next returned nil segment Jul 8 05:00:53.771144706 (355h49m20.697439396s) [upload=284997]: Waiting for error group managing segments... Jul 8 05:00:53.780217385 (355h49m20.70651207s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.780770161 (355h49m20.70706484s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781213801 (355h49m20.707508488s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781712476 (355h49m20.708007155s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.782192975 (355h49m20.708487647s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:01:23.612230509 (355h49m50.538525197s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612301272 (355h49m50.538595951s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 159.223.143.39:45850->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612349516 (355h49m50.538644191s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 159.223.143.39:38418->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612454902 (355h49m50.538749592s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612531645 (355h49m50.538826322s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 159.223.143.39:39432->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612552075 (355h49m50.538846753s) [noise=true , address=45.91.116.11:4118 , piece_id=KFPPGPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 159.223.143.39:33394->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612586657 (355h49m50.538881334s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 159.223.143.39:49070->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612604791 (355h49m50.538899468s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612642895 (355h49m50.538937572s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612651542 (355h49m50.538946218s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 159.223.143.39:47692->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612680902 (355h49m50.538975589s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612713168 (355h49m50.539007856s) [seg_pos={0 0}, upload=284997]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 8 05:01:23.648975097 (355h49m50.575269777s) [upload=284997]: Done uploading segments... Jul 8 05:01:23.649010451 (355h49m50.575305138s) [upload=284997]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 443.645754ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [............................XXX.............................] duration: 669.586899ms FAIL (2025-07-08 05:23:31 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 8) ======LOGS====== pload=284991]: Uploading piece... Jul 8 05:00:37.137063656 (355h49m4.063358349s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.137984339 (355h49m4.064279032s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138034396 (355h49m4.064329092s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138836689 (355h49m4.065131384s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138884136 (355h49m4.065178829s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139428483 (355h49m4.065723178s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139436408 (355h49m4.0657311s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139788656 (355h49m4.066083348s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139920627 (355h49m4.066215325s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140071882 (355h49m4.066366572s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140353654 (355h49m4.06664834s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140415319 (355h49m4.066710015s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140681286 (355h49m4.066975982s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140973150 (355h49m4.067267847s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.144728804 (355h49m4.071023482s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.249883590 (355h49m4.176178265s) [upload=284991]: Next returned nil segment Jul 8 05:00:37.249902715 (355h49m4.176197412s) [upload=284991]: Waiting for error group managing segments... Jul 8 05:00:37.260212837 (355h49m4.186507526s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260803813 (355h49m4.187098502s) [noise=true , address=104.36.82.45:4915 , piece_id=G6IVO3DOQGN5GQKW67NX2ZKBOVTQEBA2PUNAGFDIKUI6OLPGMZHQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260834568 (355h49m4.187129264s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261367510 (355h49m4.187662188s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261455746 (355h49m4.187750422s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261903476 (355h49m4.188198155s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.278055422 (355h49m4.204350095s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.300910056 (355h49m4.227204736s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:01:37.042774308 (355h50m3.969069003s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:19LE2Lq4e6HojnMp9xQ3ySLkRe5PemXTwiQZThN5jT46bQhN6K, address:77.90.31.19:4855): protocol: rpc: dial tcp 77.90.31.19:4855: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042887419 (355h50m3.969182106s) [noise=true , address=77.90.31.16:4747 , piece_id=HV6IOGCMOHIUEYTGJMQ62V4UUNJ2MQD37SFF5Y23X2S66YYQSN3Q, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12ERdBm3XPw4s7JHeMWsVniwcqnfMtP4ZX8sBVjri6dzZtq8AF2, address:77.90.31.16:4747): protocol: rpc: dial tcp 77.90.31.16:4747: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042966233 (355h50m3.969260923s) [noise=true , address=77.90.31.19:4856 , piece_id=CR7AX6HZZGBQM5QATGLDLHQGUPZC4KW6H5CIBB4LFRTNL2WOLQFQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:1MntjJtcCDSRozKpesp6S2ZjYrMhxiQheDiPMPSCnavWBBGYt3, address:77.90.31.19:4856): protocol: rpc: dial tcp 77.90.31.19:4856: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043090813 (355h50m3.969385503s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: write tcp 159.223.143.39:47434->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043180860 (355h50m3.969475548s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12Ttdx6LHmuAt8JVC7arCDgWjm9eLadjeaEw17F12btoXuSiq9t, address:77.90.31.11:4583): protocol: write tcp 159.223.143.39:43298->77.90.31.11:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043193162 (355h50m3.969487876s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12LuFBTJg1aNmZoEdiQTtvfnfh5qnWVZFtqFX93L4fNfiYTYqJ6, address:77.90.31.18:4818): protocol: rpc: dial tcp 77.90.31.18:4818: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043283221 (355h50m3.969577895s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:126Sou5pnMm2eE2FJpv6WYX2VSqJvZZf9rFFJD6hFYDetRD28zu, address:77.90.31.12:4603): protocol: rpc: dial tcp 77.90.31.12:4603: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043307928 (355h50m3.969602654s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043494443 (355h50m3.969789132s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:168Tw7W69Tq9opDKhJ4cM8PGnzpxp9E9SxKbdVxJeF83pETqGx, address:77.90.31.17:4804): protocol: rpc: dial tcp 77.90.31.17:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043560259 (355h50m3.969854956s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: write tcp 159.223.143.39:37628->77.90.31.13:4632: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043598718 (355h50m3.969893411s) [seg_pos={0 0}, upload=284991]: Done waiting for segment. successful: 8 optimal: 14 errs: <nil> Jul 8 05:01:37.086050110 (355h50m4.012344788s) [upload=284991]: Done uploading segments... Jul 8 05:01:37.086091388 (355h50m4.012386065s) [upload=284991]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 8) Duration: 669.611473ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................XXXX................................] duration: 467.710004ms FAIL (2025-07-08 05:23:12 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== GPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674108143 (355h49m20.600402838s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674596780 (355h49m20.600891471s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675604361 (355h49m20.601899058s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675768580 (355h49m20.602063277s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676120119 (355h49m20.60241481s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676215473 (355h49m20.602510169s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676623023 (355h49m20.602917716s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676647235 (355h49m20.602941929s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676696624 (355h49m20.602991321s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677177615 (355h49m20.603472312s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677707610 (355h49m20.604002306s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678250080 (355h49m20.604544775s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678907662 (355h49m20.605202358s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679453605 (355h49m20.6057483s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679618853 (355h49m20.605913551s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.771112310 (355h49m20.697407s) [upload=284997]: Next returned nil segment Jul 8 05:00:53.771144706 (355h49m20.697439396s) [upload=284997]: Waiting for error group managing segments... Jul 8 05:00:53.780217385 (355h49m20.70651207s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.780770161 (355h49m20.70706484s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781213801 (355h49m20.707508488s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781712476 (355h49m20.708007155s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.782192975 (355h49m20.708487647s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:01:23.612230509 (355h49m50.538525197s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612301272 (355h49m50.538595951s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 159.223.143.39:45850->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612349516 (355h49m50.538644191s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 159.223.143.39:38418->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612454902 (355h49m50.538749592s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612531645 (355h49m50.538826322s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 159.223.143.39:39432->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612552075 (355h49m50.538846753s) [noise=true , address=45.91.116.11:4118 , piece_id=KFPPGPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 159.223.143.39:33394->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612586657 (355h49m50.538881334s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 159.223.143.39:49070->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612604791 (355h49m50.538899468s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612642895 (355h49m50.538937572s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612651542 (355h49m50.538946218s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 159.223.143.39:47692->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612680902 (355h49m50.538975589s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612713168 (355h49m50.539007856s) [seg_pos={0 0}, upload=284997]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 8 05:01:23.648975097 (355h49m50.575269777s) [upload=284997]: Done uploading segments... Jul 8 05:01:23.649010451 (355h49m50.575305138s) [upload=284997]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 467.745694ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........................XXXX...............................] duration: 428.512361ms FAIL (2025-07-08 05:22:54 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== GPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674108143 (355h49m20.600402838s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674596780 (355h49m20.600891471s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675604361 (355h49m20.601899058s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675768580 (355h49m20.602063277s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676120119 (355h49m20.60241481s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676215473 (355h49m20.602510169s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676623023 (355h49m20.602917716s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676647235 (355h49m20.602941929s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676696624 (355h49m20.602991321s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677177615 (355h49m20.603472312s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677707610 (355h49m20.604002306s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678250080 (355h49m20.604544775s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678907662 (355h49m20.605202358s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679453605 (355h49m20.6057483s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679618853 (355h49m20.605913551s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.771112310 (355h49m20.697407s) [upload=284997]: Next returned nil segment Jul 8 05:00:53.771144706 (355h49m20.697439396s) [upload=284997]: Waiting for error group managing segments... Jul 8 05:00:53.780217385 (355h49m20.70651207s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.780770161 (355h49m20.70706484s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781213801 (355h49m20.707508488s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781712476 (355h49m20.708007155s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.782192975 (355h49m20.708487647s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:01:23.612230509 (355h49m50.538525197s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612301272 (355h49m50.538595951s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 159.223.143.39:45850->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612349516 (355h49m50.538644191s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 159.223.143.39:38418->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612454902 (355h49m50.538749592s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612531645 (355h49m50.538826322s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 159.223.143.39:39432->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612552075 (355h49m50.538846753s) [noise=true , address=45.91.116.11:4118 , piece_id=KFPPGPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 159.223.143.39:33394->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612586657 (355h49m50.538881334s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 159.223.143.39:49070->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612604791 (355h49m50.538899468s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612642895 (355h49m50.538937572s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612651542 (355h49m50.538946218s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 159.223.143.39:47692->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612680902 (355h49m50.538975589s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612713168 (355h49m50.539007856s) [seg_pos={0 0}, upload=284997]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 8 05:01:23.648975097 (355h49m50.575269777s) [upload=284997]: Done uploading segments... Jul 8 05:01:23.649010451 (355h49m50.575305138s) [upload=284997]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 428.536755ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.............................XXX............................] duration: 650.219917ms FAIL (2025-07-08 05:22:18 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 8) ======LOGS====== pload=284991]: Uploading piece... Jul 8 05:00:37.137063656 (355h49m4.063358349s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.137984339 (355h49m4.064279032s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138034396 (355h49m4.064329092s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138836689 (355h49m4.065131384s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138884136 (355h49m4.065178829s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139428483 (355h49m4.065723178s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139436408 (355h49m4.0657311s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139788656 (355h49m4.066083348s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139920627 (355h49m4.066215325s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140071882 (355h49m4.066366572s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140353654 (355h49m4.06664834s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140415319 (355h49m4.066710015s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140681286 (355h49m4.066975982s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140973150 (355h49m4.067267847s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.144728804 (355h49m4.071023482s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.249883590 (355h49m4.176178265s) [upload=284991]: Next returned nil segment Jul 8 05:00:37.249902715 (355h49m4.176197412s) [upload=284991]: Waiting for error group managing segments... Jul 8 05:00:37.260212837 (355h49m4.186507526s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260803813 (355h49m4.187098502s) [noise=true , address=104.36.82.45:4915 , piece_id=G6IVO3DOQGN5GQKW67NX2ZKBOVTQEBA2PUNAGFDIKUI6OLPGMZHQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260834568 (355h49m4.187129264s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261367510 (355h49m4.187662188s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261455746 (355h49m4.187750422s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261903476 (355h49m4.188198155s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.278055422 (355h49m4.204350095s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.300910056 (355h49m4.227204736s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:01:37.042774308 (355h50m3.969069003s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:19LE2Lq4e6HojnMp9xQ3ySLkRe5PemXTwiQZThN5jT46bQhN6K, address:77.90.31.19:4855): protocol: rpc: dial tcp 77.90.31.19:4855: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042887419 (355h50m3.969182106s) [noise=true , address=77.90.31.16:4747 , piece_id=HV6IOGCMOHIUEYTGJMQ62V4UUNJ2MQD37SFF5Y23X2S66YYQSN3Q, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12ERdBm3XPw4s7JHeMWsVniwcqnfMtP4ZX8sBVjri6dzZtq8AF2, address:77.90.31.16:4747): protocol: rpc: dial tcp 77.90.31.16:4747: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042966233 (355h50m3.969260923s) [noise=true , address=77.90.31.19:4856 , piece_id=CR7AX6HZZGBQM5QATGLDLHQGUPZC4KW6H5CIBB4LFRTNL2WOLQFQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:1MntjJtcCDSRozKpesp6S2ZjYrMhxiQheDiPMPSCnavWBBGYt3, address:77.90.31.19:4856): protocol: rpc: dial tcp 77.90.31.19:4856: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043090813 (355h50m3.969385503s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: write tcp 159.223.143.39:47434->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043180860 (355h50m3.969475548s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12Ttdx6LHmuAt8JVC7arCDgWjm9eLadjeaEw17F12btoXuSiq9t, address:77.90.31.11:4583): protocol: write tcp 159.223.143.39:43298->77.90.31.11:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043193162 (355h50m3.969487876s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12LuFBTJg1aNmZoEdiQTtvfnfh5qnWVZFtqFX93L4fNfiYTYqJ6, address:77.90.31.18:4818): protocol: rpc: dial tcp 77.90.31.18:4818: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043283221 (355h50m3.969577895s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:126Sou5pnMm2eE2FJpv6WYX2VSqJvZZf9rFFJD6hFYDetRD28zu, address:77.90.31.12:4603): protocol: rpc: dial tcp 77.90.31.12:4603: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043307928 (355h50m3.969602654s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043494443 (355h50m3.969789132s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:168Tw7W69Tq9opDKhJ4cM8PGnzpxp9E9SxKbdVxJeF83pETqGx, address:77.90.31.17:4804): protocol: rpc: dial tcp 77.90.31.17:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043560259 (355h50m3.969854956s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: write tcp 159.223.143.39:37628->77.90.31.13:4632: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043598718 (355h50m3.969893411s) [seg_pos={0 0}, upload=284991]: Done waiting for segment. successful: 8 optimal: 14 errs: <nil> Jul 8 05:01:37.086050110 (355h50m4.012344788s) [upload=284991]: Done uploading segments... Jul 8 05:01:37.086091388 (355h50m4.012386065s) [upload=284991]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 8) Duration: 650.245316ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........................XXXX..............................] duration: 413.874642ms FAIL (2025-07-08 05:21:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== GPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674108143 (355h49m20.600402838s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674596780 (355h49m20.600891471s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675604361 (355h49m20.601899058s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675768580 (355h49m20.602063277s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676120119 (355h49m20.60241481s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676215473 (355h49m20.602510169s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676623023 (355h49m20.602917716s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676647235 (355h49m20.602941929s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676696624 (355h49m20.602991321s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677177615 (355h49m20.603472312s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677707610 (355h49m20.604002306s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678250080 (355h49m20.604544775s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678907662 (355h49m20.605202358s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679453605 (355h49m20.6057483s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679618853 (355h49m20.605913551s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.771112310 (355h49m20.697407s) [upload=284997]: Next returned nil segment Jul 8 05:00:53.771144706 (355h49m20.697439396s) [upload=284997]: Waiting for error group managing segments... Jul 8 05:00:53.780217385 (355h49m20.70651207s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.780770161 (355h49m20.70706484s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781213801 (355h49m20.707508488s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781712476 (355h49m20.708007155s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.782192975 (355h49m20.708487647s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:01:23.612230509 (355h49m50.538525197s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612301272 (355h49m50.538595951s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 159.223.143.39:45850->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612349516 (355h49m50.538644191s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 159.223.143.39:38418->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612454902 (355h49m50.538749592s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612531645 (355h49m50.538826322s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 159.223.143.39:39432->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612552075 (355h49m50.538846753s) [noise=true , address=45.91.116.11:4118 , piece_id=KFPPGPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 159.223.143.39:33394->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612586657 (355h49m50.538881334s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 159.223.143.39:49070->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612604791 (355h49m50.538899468s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612642895 (355h49m50.538937572s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612651542 (355h49m50.538946218s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 159.223.143.39:47692->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612680902 (355h49m50.538975589s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612713168 (355h49m50.539007856s) [seg_pos={0 0}, upload=284997]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 8 05:01:23.648975097 (355h49m50.575269777s) [upload=284997]: Done uploading segments... Jul 8 05:01:23.649010451 (355h49m50.575305138s) [upload=284997]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 413.904402ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [..............................XXX...........................] duration: 481.120178ms FAIL (2025-07-08 05:21:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 8) ======LOGS====== pload=284991]: Uploading piece... Jul 8 05:00:37.137063656 (355h49m4.063358349s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.137984339 (355h49m4.064279032s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138034396 (355h49m4.064329092s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138836689 (355h49m4.065131384s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138884136 (355h49m4.065178829s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139428483 (355h49m4.065723178s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139436408 (355h49m4.0657311s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139788656 (355h49m4.066083348s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139920627 (355h49m4.066215325s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140071882 (355h49m4.066366572s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140353654 (355h49m4.06664834s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140415319 (355h49m4.066710015s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140681286 (355h49m4.066975982s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140973150 (355h49m4.067267847s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.144728804 (355h49m4.071023482s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.249883590 (355h49m4.176178265s) [upload=284991]: Next returned nil segment Jul 8 05:00:37.249902715 (355h49m4.176197412s) [upload=284991]: Waiting for error group managing segments... Jul 8 05:00:37.260212837 (355h49m4.186507526s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260803813 (355h49m4.187098502s) [noise=true , address=104.36.82.45:4915 , piece_id=G6IVO3DOQGN5GQKW67NX2ZKBOVTQEBA2PUNAGFDIKUI6OLPGMZHQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260834568 (355h49m4.187129264s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261367510 (355h49m4.187662188s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261455746 (355h49m4.187750422s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261903476 (355h49m4.188198155s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.278055422 (355h49m4.204350095s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.300910056 (355h49m4.227204736s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:01:37.042774308 (355h50m3.969069003s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:19LE2Lq4e6HojnMp9xQ3ySLkRe5PemXTwiQZThN5jT46bQhN6K, address:77.90.31.19:4855): protocol: rpc: dial tcp 77.90.31.19:4855: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042887419 (355h50m3.969182106s) [noise=true , address=77.90.31.16:4747 , piece_id=HV6IOGCMOHIUEYTGJMQ62V4UUNJ2MQD37SFF5Y23X2S66YYQSN3Q, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12ERdBm3XPw4s7JHeMWsVniwcqnfMtP4ZX8sBVjri6dzZtq8AF2, address:77.90.31.16:4747): protocol: rpc: dial tcp 77.90.31.16:4747: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042966233 (355h50m3.969260923s) [noise=true , address=77.90.31.19:4856 , piece_id=CR7AX6HZZGBQM5QATGLDLHQGUPZC4KW6H5CIBB4LFRTNL2WOLQFQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:1MntjJtcCDSRozKpesp6S2ZjYrMhxiQheDiPMPSCnavWBBGYt3, address:77.90.31.19:4856): protocol: rpc: dial tcp 77.90.31.19:4856: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043090813 (355h50m3.969385503s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: write tcp 159.223.143.39:47434->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043180860 (355h50m3.969475548s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12Ttdx6LHmuAt8JVC7arCDgWjm9eLadjeaEw17F12btoXuSiq9t, address:77.90.31.11:4583): protocol: write tcp 159.223.143.39:43298->77.90.31.11:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043193162 (355h50m3.969487876s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12LuFBTJg1aNmZoEdiQTtvfnfh5qnWVZFtqFX93L4fNfiYTYqJ6, address:77.90.31.18:4818): protocol: rpc: dial tcp 77.90.31.18:4818: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043283221 (355h50m3.969577895s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:126Sou5pnMm2eE2FJpv6WYX2VSqJvZZf9rFFJD6hFYDetRD28zu, address:77.90.31.12:4603): protocol: rpc: dial tcp 77.90.31.12:4603: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043307928 (355h50m3.969602654s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043494443 (355h50m3.969789132s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:168Tw7W69Tq9opDKhJ4cM8PGnzpxp9E9SxKbdVxJeF83pETqGx, address:77.90.31.17:4804): protocol: rpc: dial tcp 77.90.31.17:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043560259 (355h50m3.969854956s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: write tcp 159.223.143.39:37628->77.90.31.13:4632: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043598718 (355h50m3.969893411s) [seg_pos={0 0}, upload=284991]: Done waiting for segment. successful: 8 optimal: 14 errs: <nil> Jul 8 05:01:37.086050110 (355h50m4.012344788s) [upload=284991]: Done uploading segments... Jul 8 05:01:37.086091388 (355h50m4.012386065s) [upload=284991]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 8) Duration: 481.149226ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........................XXXX.............................] duration: 443.921449ms FAIL (2025-07-08 05:20:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== GPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674108143 (355h49m20.600402838s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674596780 (355h49m20.600891471s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675604361 (355h49m20.601899058s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675768580 (355h49m20.602063277s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676120119 (355h49m20.60241481s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676215473 (355h49m20.602510169s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676623023 (355h49m20.602917716s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676647235 (355h49m20.602941929s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676696624 (355h49m20.602991321s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677177615 (355h49m20.603472312s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677707610 (355h49m20.604002306s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678250080 (355h49m20.604544775s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678907662 (355h49m20.605202358s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679453605 (355h49m20.6057483s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679618853 (355h49m20.605913551s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.771112310 (355h49m20.697407s) [upload=284997]: Next returned nil segment Jul 8 05:00:53.771144706 (355h49m20.697439396s) [upload=284997]: Waiting for error group managing segments... Jul 8 05:00:53.780217385 (355h49m20.70651207s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.780770161 (355h49m20.70706484s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781213801 (355h49m20.707508488s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781712476 (355h49m20.708007155s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.782192975 (355h49m20.708487647s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:01:23.612230509 (355h49m50.538525197s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612301272 (355h49m50.538595951s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 159.223.143.39:45850->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612349516 (355h49m50.538644191s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 159.223.143.39:38418->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612454902 (355h49m50.538749592s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612531645 (355h49m50.538826322s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 159.223.143.39:39432->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612552075 (355h49m50.538846753s) [noise=true , address=45.91.116.11:4118 , piece_id=KFPPGPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 159.223.143.39:33394->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612586657 (355h49m50.538881334s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 159.223.143.39:49070->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612604791 (355h49m50.538899468s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612642895 (355h49m50.538937572s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612651542 (355h49m50.538946218s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 159.223.143.39:47692->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612680902 (355h49m50.538975589s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612713168 (355h49m50.539007856s) [seg_pos={0 0}, upload=284997]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 8 05:01:23.648975097 (355h49m50.575269777s) [upload=284997]: Done uploading segments... Jul 8 05:01:23.649010451 (355h49m50.575305138s) [upload=284997]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 443.954921ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [...............................XXX..........................] duration: 649.956917ms FAIL (2025-07-08 05:20:31 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 8) ======LOGS====== pload=284991]: Uploading piece... Jul 8 05:00:37.137063656 (355h49m4.063358349s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.137984339 (355h49m4.064279032s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138034396 (355h49m4.064329092s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138836689 (355h49m4.065131384s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138884136 (355h49m4.065178829s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139428483 (355h49m4.065723178s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139436408 (355h49m4.0657311s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139788656 (355h49m4.066083348s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139920627 (355h49m4.066215325s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140071882 (355h49m4.066366572s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140353654 (355h49m4.06664834s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140415319 (355h49m4.066710015s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140681286 (355h49m4.066975982s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140973150 (355h49m4.067267847s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.144728804 (355h49m4.071023482s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.249883590 (355h49m4.176178265s) [upload=284991]: Next returned nil segment Jul 8 05:00:37.249902715 (355h49m4.176197412s) [upload=284991]: Waiting for error group managing segments... Jul 8 05:00:37.260212837 (355h49m4.186507526s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260803813 (355h49m4.187098502s) [noise=true , address=104.36.82.45:4915 , piece_id=G6IVO3DOQGN5GQKW67NX2ZKBOVTQEBA2PUNAGFDIKUI6OLPGMZHQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260834568 (355h49m4.187129264s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261367510 (355h49m4.187662188s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261455746 (355h49m4.187750422s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261903476 (355h49m4.188198155s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.278055422 (355h49m4.204350095s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.300910056 (355h49m4.227204736s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:01:37.042774308 (355h50m3.969069003s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:19LE2Lq4e6HojnMp9xQ3ySLkRe5PemXTwiQZThN5jT46bQhN6K, address:77.90.31.19:4855): protocol: rpc: dial tcp 77.90.31.19:4855: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042887419 (355h50m3.969182106s) [noise=true , address=77.90.31.16:4747 , piece_id=HV6IOGCMOHIUEYTGJMQ62V4UUNJ2MQD37SFF5Y23X2S66YYQSN3Q, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12ERdBm3XPw4s7JHeMWsVniwcqnfMtP4ZX8sBVjri6dzZtq8AF2, address:77.90.31.16:4747): protocol: rpc: dial tcp 77.90.31.16:4747: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042966233 (355h50m3.969260923s) [noise=true , address=77.90.31.19:4856 , piece_id=CR7AX6HZZGBQM5QATGLDLHQGUPZC4KW6H5CIBB4LFRTNL2WOLQFQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:1MntjJtcCDSRozKpesp6S2ZjYrMhxiQheDiPMPSCnavWBBGYt3, address:77.90.31.19:4856): protocol: rpc: dial tcp 77.90.31.19:4856: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043090813 (355h50m3.969385503s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: write tcp 159.223.143.39:47434->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043180860 (355h50m3.969475548s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12Ttdx6LHmuAt8JVC7arCDgWjm9eLadjeaEw17F12btoXuSiq9t, address:77.90.31.11:4583): protocol: write tcp 159.223.143.39:43298->77.90.31.11:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043193162 (355h50m3.969487876s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12LuFBTJg1aNmZoEdiQTtvfnfh5qnWVZFtqFX93L4fNfiYTYqJ6, address:77.90.31.18:4818): protocol: rpc: dial tcp 77.90.31.18:4818: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043283221 (355h50m3.969577895s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:126Sou5pnMm2eE2FJpv6WYX2VSqJvZZf9rFFJD6hFYDetRD28zu, address:77.90.31.12:4603): protocol: rpc: dial tcp 77.90.31.12:4603: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043307928 (355h50m3.969602654s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043494443 (355h50m3.969789132s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:168Tw7W69Tq9opDKhJ4cM8PGnzpxp9E9SxKbdVxJeF83pETqGx, address:77.90.31.17:4804): protocol: rpc: dial tcp 77.90.31.17:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043560259 (355h50m3.969854956s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: write tcp 159.223.143.39:37628->77.90.31.13:4632: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043598718 (355h50m3.969893411s) [seg_pos={0 0}, upload=284991]: Done waiting for segment. successful: 8 optimal: 14 errs: <nil> Jul 8 05:01:37.086050110 (355h50m4.012344788s) [upload=284991]: Done uploading segments... Jul 8 05:01:37.086091388 (355h50m4.012386065s) [upload=284991]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 8) Duration: 650.09347ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [............................XXXX............................] duration: 440.271166ms FAIL (2025-07-08 05:20:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== GPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674108143 (355h49m20.600402838s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674596780 (355h49m20.600891471s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675604361 (355h49m20.601899058s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675768580 (355h49m20.602063277s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676120119 (355h49m20.60241481s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676215473 (355h49m20.602510169s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676623023 (355h49m20.602917716s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676647235 (355h49m20.602941929s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676696624 (355h49m20.602991321s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677177615 (355h49m20.603472312s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677707610 (355h49m20.604002306s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678250080 (355h49m20.604544775s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678907662 (355h49m20.605202358s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679453605 (355h49m20.6057483s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679618853 (355h49m20.605913551s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.771112310 (355h49m20.697407s) [upload=284997]: Next returned nil segment Jul 8 05:00:53.771144706 (355h49m20.697439396s) [upload=284997]: Waiting for error group managing segments... Jul 8 05:00:53.780217385 (355h49m20.70651207s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.780770161 (355h49m20.70706484s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781213801 (355h49m20.707508488s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781712476 (355h49m20.708007155s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.782192975 (355h49m20.708487647s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:01:23.612230509 (355h49m50.538525197s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612301272 (355h49m50.538595951s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 159.223.143.39:45850->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612349516 (355h49m50.538644191s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 159.223.143.39:38418->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612454902 (355h49m50.538749592s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612531645 (355h49m50.538826322s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 159.223.143.39:39432->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612552075 (355h49m50.538846753s) [noise=true , address=45.91.116.11:4118 , piece_id=KFPPGPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 159.223.143.39:33394->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612586657 (355h49m50.538881334s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 159.223.143.39:49070->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612604791 (355h49m50.538899468s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612642895 (355h49m50.538937572s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612651542 (355h49m50.538946218s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 159.223.143.39:47692->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612680902 (355h49m50.538975589s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612713168 (355h49m50.539007856s) [seg_pos={0 0}, upload=284997]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 8 05:01:23.648975097 (355h49m50.575269777s) [upload=284997]: Done uploading segments... Jul 8 05:01:23.649010451 (355h49m50.575305138s) [upload=284997]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 440.301419ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............................XXXX...........................] duration: 428.464863ms FAIL (2025-07-08 05:19:42 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== GPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674108143 (355h49m20.600402838s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674596780 (355h49m20.600891471s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675604361 (355h49m20.601899058s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675768580 (355h49m20.602063277s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676120119 (355h49m20.60241481s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676215473 (355h49m20.602510169s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676623023 (355h49m20.602917716s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676647235 (355h49m20.602941929s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676696624 (355h49m20.602991321s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677177615 (355h49m20.603472312s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677707610 (355h49m20.604002306s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678250080 (355h49m20.604544775s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678907662 (355h49m20.605202358s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679453605 (355h49m20.6057483s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679618853 (355h49m20.605913551s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.771112310 (355h49m20.697407s) [upload=284997]: Next returned nil segment Jul 8 05:00:53.771144706 (355h49m20.697439396s) [upload=284997]: Waiting for error group managing segments... Jul 8 05:00:53.780217385 (355h49m20.70651207s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.780770161 (355h49m20.70706484s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781213801 (355h49m20.707508488s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781712476 (355h49m20.708007155s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.782192975 (355h49m20.708487647s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:01:23.612230509 (355h49m50.538525197s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612301272 (355h49m50.538595951s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 159.223.143.39:45850->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612349516 (355h49m50.538644191s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 159.223.143.39:38418->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612454902 (355h49m50.538749592s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612531645 (355h49m50.538826322s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 159.223.143.39:39432->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612552075 (355h49m50.538846753s) [noise=true , address=45.91.116.11:4118 , piece_id=KFPPGPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 159.223.143.39:33394->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612586657 (355h49m50.538881334s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 159.223.143.39:49070->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612604791 (355h49m50.538899468s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612642895 (355h49m50.538937572s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612651542 (355h49m50.538946218s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 159.223.143.39:47692->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612680902 (355h49m50.538975589s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612713168 (355h49m50.539007856s) [seg_pos={0 0}, upload=284997]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 8 05:01:23.648975097 (355h49m50.575269777s) [upload=284997]: Done uploading segments... Jul 8 05:01:23.649010451 (355h49m50.575305138s) [upload=284997]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 428.499258ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [................................XXX.........................] duration: 416.108723ms FAIL (2025-07-08 05:19:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 8) ======LOGS====== pload=284991]: Uploading piece... Jul 8 05:00:37.137063656 (355h49m4.063358349s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.137984339 (355h49m4.064279032s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138034396 (355h49m4.064329092s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138836689 (355h49m4.065131384s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138884136 (355h49m4.065178829s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139428483 (355h49m4.065723178s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139436408 (355h49m4.0657311s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139788656 (355h49m4.066083348s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139920627 (355h49m4.066215325s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140071882 (355h49m4.066366572s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140353654 (355h49m4.06664834s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140415319 (355h49m4.066710015s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140681286 (355h49m4.066975982s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140973150 (355h49m4.067267847s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.144728804 (355h49m4.071023482s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.249883590 (355h49m4.176178265s) [upload=284991]: Next returned nil segment Jul 8 05:00:37.249902715 (355h49m4.176197412s) [upload=284991]: Waiting for error group managing segments... Jul 8 05:00:37.260212837 (355h49m4.186507526s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260803813 (355h49m4.187098502s) [noise=true , address=104.36.82.45:4915 , piece_id=G6IVO3DOQGN5GQKW67NX2ZKBOVTQEBA2PUNAGFDIKUI6OLPGMZHQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260834568 (355h49m4.187129264s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261367510 (355h49m4.187662188s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261455746 (355h49m4.187750422s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261903476 (355h49m4.188198155s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.278055422 (355h49m4.204350095s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.300910056 (355h49m4.227204736s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:01:37.042774308 (355h50m3.969069003s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:19LE2Lq4e6HojnMp9xQ3ySLkRe5PemXTwiQZThN5jT46bQhN6K, address:77.90.31.19:4855): protocol: rpc: dial tcp 77.90.31.19:4855: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042887419 (355h50m3.969182106s) [noise=true , address=77.90.31.16:4747 , piece_id=HV6IOGCMOHIUEYTGJMQ62V4UUNJ2MQD37SFF5Y23X2S66YYQSN3Q, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12ERdBm3XPw4s7JHeMWsVniwcqnfMtP4ZX8sBVjri6dzZtq8AF2, address:77.90.31.16:4747): protocol: rpc: dial tcp 77.90.31.16:4747: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042966233 (355h50m3.969260923s) [noise=true , address=77.90.31.19:4856 , piece_id=CR7AX6HZZGBQM5QATGLDLHQGUPZC4KW6H5CIBB4LFRTNL2WOLQFQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:1MntjJtcCDSRozKpesp6S2ZjYrMhxiQheDiPMPSCnavWBBGYt3, address:77.90.31.19:4856): protocol: rpc: dial tcp 77.90.31.19:4856: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043090813 (355h50m3.969385503s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: write tcp 159.223.143.39:47434->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043180860 (355h50m3.969475548s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12Ttdx6LHmuAt8JVC7arCDgWjm9eLadjeaEw17F12btoXuSiq9t, address:77.90.31.11:4583): protocol: write tcp 159.223.143.39:43298->77.90.31.11:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043193162 (355h50m3.969487876s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12LuFBTJg1aNmZoEdiQTtvfnfh5qnWVZFtqFX93L4fNfiYTYqJ6, address:77.90.31.18:4818): protocol: rpc: dial tcp 77.90.31.18:4818: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043283221 (355h50m3.969577895s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:126Sou5pnMm2eE2FJpv6WYX2VSqJvZZf9rFFJD6hFYDetRD28zu, address:77.90.31.12:4603): protocol: rpc: dial tcp 77.90.31.12:4603: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043307928 (355h50m3.969602654s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043494443 (355h50m3.969789132s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:168Tw7W69Tq9opDKhJ4cM8PGnzpxp9E9SxKbdVxJeF83pETqGx, address:77.90.31.17:4804): protocol: rpc: dial tcp 77.90.31.17:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043560259 (355h50m3.969854956s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: write tcp 159.223.143.39:37628->77.90.31.13:4632: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043598718 (355h50m3.969893411s) [seg_pos={0 0}, upload=284991]: Done waiting for segment. successful: 8 optimal: 14 errs: <nil> Jul 8 05:01:37.086050110 (355h50m4.012344788s) [upload=284991]: Done uploading segments... Jul 8 05:01:37.086091388 (355h50m4.012386065s) [upload=284991]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 8) Duration: 416.141339ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.................................XXX........................] duration: 641.655724ms FAIL (2025-07-08 05:18:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 8) ======LOGS====== pload=284991]: Uploading piece... Jul 8 05:00:37.137063656 (355h49m4.063358349s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.137984339 (355h49m4.064279032s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138034396 (355h49m4.064329092s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138836689 (355h49m4.065131384s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138884136 (355h49m4.065178829s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139428483 (355h49m4.065723178s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139436408 (355h49m4.0657311s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139788656 (355h49m4.066083348s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139920627 (355h49m4.066215325s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140071882 (355h49m4.066366572s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140353654 (355h49m4.06664834s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140415319 (355h49m4.066710015s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140681286 (355h49m4.066975982s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140973150 (355h49m4.067267847s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.144728804 (355h49m4.071023482s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.249883590 (355h49m4.176178265s) [upload=284991]: Next returned nil segment Jul 8 05:00:37.249902715 (355h49m4.176197412s) [upload=284991]: Waiting for error group managing segments... Jul 8 05:00:37.260212837 (355h49m4.186507526s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260803813 (355h49m4.187098502s) [noise=true , address=104.36.82.45:4915 , piece_id=G6IVO3DOQGN5GQKW67NX2ZKBOVTQEBA2PUNAGFDIKUI6OLPGMZHQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260834568 (355h49m4.187129264s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261367510 (355h49m4.187662188s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261455746 (355h49m4.187750422s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261903476 (355h49m4.188198155s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.278055422 (355h49m4.204350095s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.300910056 (355h49m4.227204736s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:01:37.042774308 (355h50m3.969069003s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:19LE2Lq4e6HojnMp9xQ3ySLkRe5PemXTwiQZThN5jT46bQhN6K, address:77.90.31.19:4855): protocol: rpc: dial tcp 77.90.31.19:4855: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042887419 (355h50m3.969182106s) [noise=true , address=77.90.31.16:4747 , piece_id=HV6IOGCMOHIUEYTGJMQ62V4UUNJ2MQD37SFF5Y23X2S66YYQSN3Q, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12ERdBm3XPw4s7JHeMWsVniwcqnfMtP4ZX8sBVjri6dzZtq8AF2, address:77.90.31.16:4747): protocol: rpc: dial tcp 77.90.31.16:4747: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042966233 (355h50m3.969260923s) [noise=true , address=77.90.31.19:4856 , piece_id=CR7AX6HZZGBQM5QATGLDLHQGUPZC4KW6H5CIBB4LFRTNL2WOLQFQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:1MntjJtcCDSRozKpesp6S2ZjYrMhxiQheDiPMPSCnavWBBGYt3, address:77.90.31.19:4856): protocol: rpc: dial tcp 77.90.31.19:4856: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043090813 (355h50m3.969385503s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: write tcp 159.223.143.39:47434->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043180860 (355h50m3.969475548s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12Ttdx6LHmuAt8JVC7arCDgWjm9eLadjeaEw17F12btoXuSiq9t, address:77.90.31.11:4583): protocol: write tcp 159.223.143.39:43298->77.90.31.11:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043193162 (355h50m3.969487876s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12LuFBTJg1aNmZoEdiQTtvfnfh5qnWVZFtqFX93L4fNfiYTYqJ6, address:77.90.31.18:4818): protocol: rpc: dial tcp 77.90.31.18:4818: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043283221 (355h50m3.969577895s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:126Sou5pnMm2eE2FJpv6WYX2VSqJvZZf9rFFJD6hFYDetRD28zu, address:77.90.31.12:4603): protocol: rpc: dial tcp 77.90.31.12:4603: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043307928 (355h50m3.969602654s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043494443 (355h50m3.969789132s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:168Tw7W69Tq9opDKhJ4cM8PGnzpxp9E9SxKbdVxJeF83pETqGx, address:77.90.31.17:4804): protocol: rpc: dial tcp 77.90.31.17:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043560259 (355h50m3.969854956s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: write tcp 159.223.143.39:37628->77.90.31.13:4632: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043598718 (355h50m3.969893411s) [seg_pos={0 0}, upload=284991]: Done waiting for segment. successful: 8 optimal: 14 errs: <nil> Jul 8 05:01:37.086050110 (355h50m4.012344788s) [upload=284991]: Done uploading segments... Jul 8 05:01:37.086091388 (355h50m4.012386065s) [upload=284991]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 8) Duration: 641.688984ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............................XXXX..........................] duration: 461.071227ms FAIL (2025-07-08 05:18:46 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== GPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674108143 (355h49m20.600402838s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674596780 (355h49m20.600891471s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675604361 (355h49m20.601899058s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675768580 (355h49m20.602063277s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676120119 (355h49m20.60241481s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676215473 (355h49m20.602510169s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676623023 (355h49m20.602917716s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676647235 (355h49m20.602941929s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676696624 (355h49m20.602991321s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677177615 (355h49m20.603472312s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677707610 (355h49m20.604002306s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678250080 (355h49m20.604544775s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678907662 (355h49m20.605202358s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679453605 (355h49m20.6057483s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679618853 (355h49m20.605913551s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.771112310 (355h49m20.697407s) [upload=284997]: Next returned nil segment Jul 8 05:00:53.771144706 (355h49m20.697439396s) [upload=284997]: Waiting for error group managing segments... Jul 8 05:00:53.780217385 (355h49m20.70651207s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.780770161 (355h49m20.70706484s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781213801 (355h49m20.707508488s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781712476 (355h49m20.708007155s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.782192975 (355h49m20.708487647s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:01:23.612230509 (355h49m50.538525197s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612301272 (355h49m50.538595951s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 159.223.143.39:45850->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612349516 (355h49m50.538644191s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 159.223.143.39:38418->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612454902 (355h49m50.538749592s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612531645 (355h49m50.538826322s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 159.223.143.39:39432->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612552075 (355h49m50.538846753s) [noise=true , address=45.91.116.11:4118 , piece_id=KFPPGPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 159.223.143.39:33394->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612586657 (355h49m50.538881334s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 159.223.143.39:49070->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612604791 (355h49m50.538899468s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612642895 (355h49m50.538937572s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612651542 (355h49m50.538946218s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 159.223.143.39:47692->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612680902 (355h49m50.538975589s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612713168 (355h49m50.539007856s) [seg_pos={0 0}, upload=284997]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 8 05:01:23.648975097 (355h49m50.575269777s) [upload=284997]: Done uploading segments... Jul 8 05:01:23.649010451 (355h49m50.575305138s) [upload=284997]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 461.09832ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............................XXXX.........................] duration: 444.309769ms FAIL (2025-07-08 05:18:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== GPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674108143 (355h49m20.600402838s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674596780 (355h49m20.600891471s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675604361 (355h49m20.601899058s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675768580 (355h49m20.602063277s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676120119 (355h49m20.60241481s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676215473 (355h49m20.602510169s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676623023 (355h49m20.602917716s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676647235 (355h49m20.602941929s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676696624 (355h49m20.602991321s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677177615 (355h49m20.603472312s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677707610 (355h49m20.604002306s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678250080 (355h49m20.604544775s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678907662 (355h49m20.605202358s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679453605 (355h49m20.6057483s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679618853 (355h49m20.605913551s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.771112310 (355h49m20.697407s) [upload=284997]: Next returned nil segment Jul 8 05:00:53.771144706 (355h49m20.697439396s) [upload=284997]: Waiting for error group managing segments... Jul 8 05:00:53.780217385 (355h49m20.70651207s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.780770161 (355h49m20.70706484s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781213801 (355h49m20.707508488s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781712476 (355h49m20.708007155s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.782192975 (355h49m20.708487647s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:01:23.612230509 (355h49m50.538525197s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612301272 (355h49m50.538595951s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 159.223.143.39:45850->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612349516 (355h49m50.538644191s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 159.223.143.39:38418->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612454902 (355h49m50.538749592s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612531645 (355h49m50.538826322s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 159.223.143.39:39432->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612552075 (355h49m50.538846753s) [noise=true , address=45.91.116.11:4118 , piece_id=KFPPGPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 159.223.143.39:33394->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612586657 (355h49m50.538881334s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 159.223.143.39:49070->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612604791 (355h49m50.538899468s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612642895 (355h49m50.538937572s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612651542 (355h49m50.538946218s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 159.223.143.39:47692->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612680902 (355h49m50.538975589s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612713168 (355h49m50.539007856s) [seg_pos={0 0}, upload=284997]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 8 05:01:23.648975097 (355h49m50.575269777s) [upload=284997]: Done uploading segments... Jul 8 05:01:23.649010451 (355h49m50.575305138s) [upload=284997]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 444.34422ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [..................................XXX.......................] duration: 434.995854ms FAIL (2025-07-08 05:18:17 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 8) ======LOGS====== pload=284991]: Uploading piece... Jul 8 05:00:37.137063656 (355h49m4.063358349s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.137984339 (355h49m4.064279032s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138034396 (355h49m4.064329092s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138836689 (355h49m4.065131384s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138884136 (355h49m4.065178829s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139428483 (355h49m4.065723178s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139436408 (355h49m4.0657311s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139788656 (355h49m4.066083348s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139920627 (355h49m4.066215325s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140071882 (355h49m4.066366572s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140353654 (355h49m4.06664834s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140415319 (355h49m4.066710015s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140681286 (355h49m4.066975982s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140973150 (355h49m4.067267847s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.144728804 (355h49m4.071023482s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.249883590 (355h49m4.176178265s) [upload=284991]: Next returned nil segment Jul 8 05:00:37.249902715 (355h49m4.176197412s) [upload=284991]: Waiting for error group managing segments... Jul 8 05:00:37.260212837 (355h49m4.186507526s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260803813 (355h49m4.187098502s) [noise=true , address=104.36.82.45:4915 , piece_id=G6IVO3DOQGN5GQKW67NX2ZKBOVTQEBA2PUNAGFDIKUI6OLPGMZHQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260834568 (355h49m4.187129264s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261367510 (355h49m4.187662188s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261455746 (355h49m4.187750422s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261903476 (355h49m4.188198155s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.278055422 (355h49m4.204350095s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.300910056 (355h49m4.227204736s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:01:37.042774308 (355h50m3.969069003s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:19LE2Lq4e6HojnMp9xQ3ySLkRe5PemXTwiQZThN5jT46bQhN6K, address:77.90.31.19:4855): protocol: rpc: dial tcp 77.90.31.19:4855: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042887419 (355h50m3.969182106s) [noise=true , address=77.90.31.16:4747 , piece_id=HV6IOGCMOHIUEYTGJMQ62V4UUNJ2MQD37SFF5Y23X2S66YYQSN3Q, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12ERdBm3XPw4s7JHeMWsVniwcqnfMtP4ZX8sBVjri6dzZtq8AF2, address:77.90.31.16:4747): protocol: rpc: dial tcp 77.90.31.16:4747: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042966233 (355h50m3.969260923s) [noise=true , address=77.90.31.19:4856 , piece_id=CR7AX6HZZGBQM5QATGLDLHQGUPZC4KW6H5CIBB4LFRTNL2WOLQFQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:1MntjJtcCDSRozKpesp6S2ZjYrMhxiQheDiPMPSCnavWBBGYt3, address:77.90.31.19:4856): protocol: rpc: dial tcp 77.90.31.19:4856: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043090813 (355h50m3.969385503s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: write tcp 159.223.143.39:47434->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043180860 (355h50m3.969475548s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12Ttdx6LHmuAt8JVC7arCDgWjm9eLadjeaEw17F12btoXuSiq9t, address:77.90.31.11:4583): protocol: write tcp 159.223.143.39:43298->77.90.31.11:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043193162 (355h50m3.969487876s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12LuFBTJg1aNmZoEdiQTtvfnfh5qnWVZFtqFX93L4fNfiYTYqJ6, address:77.90.31.18:4818): protocol: rpc: dial tcp 77.90.31.18:4818: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043283221 (355h50m3.969577895s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:126Sou5pnMm2eE2FJpv6WYX2VSqJvZZf9rFFJD6hFYDetRD28zu, address:77.90.31.12:4603): protocol: rpc: dial tcp 77.90.31.12:4603: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043307928 (355h50m3.969602654s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043494443 (355h50m3.969789132s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:168Tw7W69Tq9opDKhJ4cM8PGnzpxp9E9SxKbdVxJeF83pETqGx, address:77.90.31.17:4804): protocol: rpc: dial tcp 77.90.31.17:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043560259 (355h50m3.969854956s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: write tcp 159.223.143.39:37628->77.90.31.13:4632: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043598718 (355h50m3.969893411s) [seg_pos={0 0}, upload=284991]: Done waiting for segment. successful: 8 optimal: 14 errs: <nil> Jul 8 05:01:37.086050110 (355h50m4.012344788s) [upload=284991]: Done uploading segments... Jul 8 05:01:37.086091388 (355h50m4.012386065s) [upload=284991]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 8) Duration: 435.036722ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [................................XXXX........................] duration: 405.505811ms FAIL (2025-07-08 05:17:19 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== GPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674108143 (355h49m20.600402838s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674596780 (355h49m20.600891471s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675604361 (355h49m20.601899058s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675768580 (355h49m20.602063277s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676120119 (355h49m20.60241481s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676215473 (355h49m20.602510169s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676623023 (355h49m20.602917716s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676647235 (355h49m20.602941929s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676696624 (355h49m20.602991321s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677177615 (355h49m20.603472312s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677707610 (355h49m20.604002306s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678250080 (355h49m20.604544775s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678907662 (355h49m20.605202358s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679453605 (355h49m20.6057483s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679618853 (355h49m20.605913551s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.771112310 (355h49m20.697407s) [upload=284997]: Next returned nil segment Jul 8 05:00:53.771144706 (355h49m20.697439396s) [upload=284997]: Waiting for error group managing segments... Jul 8 05:00:53.780217385 (355h49m20.70651207s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.780770161 (355h49m20.70706484s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781213801 (355h49m20.707508488s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781712476 (355h49m20.708007155s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.782192975 (355h49m20.708487647s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:01:23.612230509 (355h49m50.538525197s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612301272 (355h49m50.538595951s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 159.223.143.39:45850->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612349516 (355h49m50.538644191s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 159.223.143.39:38418->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612454902 (355h49m50.538749592s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612531645 (355h49m50.538826322s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 159.223.143.39:39432->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612552075 (355h49m50.538846753s) [noise=true , address=45.91.116.11:4118 , piece_id=KFPPGPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 159.223.143.39:33394->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612586657 (355h49m50.538881334s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 159.223.143.39:49070->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612604791 (355h49m50.538899468s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612642895 (355h49m50.538937572s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612651542 (355h49m50.538946218s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 159.223.143.39:47692->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612680902 (355h49m50.538975589s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612713168 (355h49m50.539007856s) [seg_pos={0 0}, upload=284997]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 8 05:01:23.648975097 (355h49m50.575269777s) [upload=284997]: Done uploading segments... Jul 8 05:01:23.649010451 (355h49m50.575305138s) [upload=284997]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 405.536659ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [...................................XXX......................] duration: 478.609165ms FAIL (2025-07-08 05:17:13 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 8) ======LOGS====== pload=284991]: Uploading piece... Jul 8 05:00:37.137063656 (355h49m4.063358349s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.137984339 (355h49m4.064279032s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138034396 (355h49m4.064329092s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138836689 (355h49m4.065131384s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138884136 (355h49m4.065178829s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139428483 (355h49m4.065723178s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139436408 (355h49m4.0657311s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139788656 (355h49m4.066083348s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139920627 (355h49m4.066215325s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140071882 (355h49m4.066366572s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140353654 (355h49m4.06664834s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140415319 (355h49m4.066710015s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140681286 (355h49m4.066975982s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140973150 (355h49m4.067267847s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.144728804 (355h49m4.071023482s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.249883590 (355h49m4.176178265s) [upload=284991]: Next returned nil segment Jul 8 05:00:37.249902715 (355h49m4.176197412s) [upload=284991]: Waiting for error group managing segments... Jul 8 05:00:37.260212837 (355h49m4.186507526s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260803813 (355h49m4.187098502s) [noise=true , address=104.36.82.45:4915 , piece_id=G6IVO3DOQGN5GQKW67NX2ZKBOVTQEBA2PUNAGFDIKUI6OLPGMZHQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260834568 (355h49m4.187129264s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261367510 (355h49m4.187662188s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261455746 (355h49m4.187750422s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261903476 (355h49m4.188198155s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.278055422 (355h49m4.204350095s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.300910056 (355h49m4.227204736s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:01:37.042774308 (355h50m3.969069003s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:19LE2Lq4e6HojnMp9xQ3ySLkRe5PemXTwiQZThN5jT46bQhN6K, address:77.90.31.19:4855): protocol: rpc: dial tcp 77.90.31.19:4855: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042887419 (355h50m3.969182106s) [noise=true , address=77.90.31.16:4747 , piece_id=HV6IOGCMOHIUEYTGJMQ62V4UUNJ2MQD37SFF5Y23X2S66YYQSN3Q, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12ERdBm3XPw4s7JHeMWsVniwcqnfMtP4ZX8sBVjri6dzZtq8AF2, address:77.90.31.16:4747): protocol: rpc: dial tcp 77.90.31.16:4747: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042966233 (355h50m3.969260923s) [noise=true , address=77.90.31.19:4856 , piece_id=CR7AX6HZZGBQM5QATGLDLHQGUPZC4KW6H5CIBB4LFRTNL2WOLQFQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:1MntjJtcCDSRozKpesp6S2ZjYrMhxiQheDiPMPSCnavWBBGYt3, address:77.90.31.19:4856): protocol: rpc: dial tcp 77.90.31.19:4856: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043090813 (355h50m3.969385503s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: write tcp 159.223.143.39:47434->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043180860 (355h50m3.969475548s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12Ttdx6LHmuAt8JVC7arCDgWjm9eLadjeaEw17F12btoXuSiq9t, address:77.90.31.11:4583): protocol: write tcp 159.223.143.39:43298->77.90.31.11:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043193162 (355h50m3.969487876s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12LuFBTJg1aNmZoEdiQTtvfnfh5qnWVZFtqFX93L4fNfiYTYqJ6, address:77.90.31.18:4818): protocol: rpc: dial tcp 77.90.31.18:4818: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043283221 (355h50m3.969577895s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:126Sou5pnMm2eE2FJpv6WYX2VSqJvZZf9rFFJD6hFYDetRD28zu, address:77.90.31.12:4603): protocol: rpc: dial tcp 77.90.31.12:4603: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043307928 (355h50m3.969602654s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043494443 (355h50m3.969789132s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:168Tw7W69Tq9opDKhJ4cM8PGnzpxp9E9SxKbdVxJeF83pETqGx, address:77.90.31.17:4804): protocol: rpc: dial tcp 77.90.31.17:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043560259 (355h50m3.969854956s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: write tcp 159.223.143.39:37628->77.90.31.13:4632: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043598718 (355h50m3.969893411s) [seg_pos={0 0}, upload=284991]: Done waiting for segment. successful: 8 optimal: 14 errs: <nil> Jul 8 05:01:37.086050110 (355h50m4.012344788s) [upload=284991]: Done uploading segments... Jul 8 05:01:37.086091388 (355h50m4.012386065s) [upload=284991]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 8) Duration: 478.757566ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [....................................XXX.....................] duration: 425.547155ms FAIL (2025-07-08 05:16:46 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 8) ======LOGS====== pload=284991]: Uploading piece... Jul 8 05:00:37.137063656 (355h49m4.063358349s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.137984339 (355h49m4.064279032s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138034396 (355h49m4.064329092s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138836689 (355h49m4.065131384s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138884136 (355h49m4.065178829s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139428483 (355h49m4.065723178s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139436408 (355h49m4.0657311s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139788656 (355h49m4.066083348s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139920627 (355h49m4.066215325s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140071882 (355h49m4.066366572s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140353654 (355h49m4.06664834s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140415319 (355h49m4.066710015s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140681286 (355h49m4.066975982s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140973150 (355h49m4.067267847s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.144728804 (355h49m4.071023482s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.249883590 (355h49m4.176178265s) [upload=284991]: Next returned nil segment Jul 8 05:00:37.249902715 (355h49m4.176197412s) [upload=284991]: Waiting for error group managing segments... Jul 8 05:00:37.260212837 (355h49m4.186507526s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260803813 (355h49m4.187098502s) [noise=true , address=104.36.82.45:4915 , piece_id=G6IVO3DOQGN5GQKW67NX2ZKBOVTQEBA2PUNAGFDIKUI6OLPGMZHQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260834568 (355h49m4.187129264s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261367510 (355h49m4.187662188s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261455746 (355h49m4.187750422s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261903476 (355h49m4.188198155s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.278055422 (355h49m4.204350095s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.300910056 (355h49m4.227204736s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:01:37.042774308 (355h50m3.969069003s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:19LE2Lq4e6HojnMp9xQ3ySLkRe5PemXTwiQZThN5jT46bQhN6K, address:77.90.31.19:4855): protocol: rpc: dial tcp 77.90.31.19:4855: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042887419 (355h50m3.969182106s) [noise=true , address=77.90.31.16:4747 , piece_id=HV6IOGCMOHIUEYTGJMQ62V4UUNJ2MQD37SFF5Y23X2S66YYQSN3Q, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12ERdBm3XPw4s7JHeMWsVniwcqnfMtP4ZX8sBVjri6dzZtq8AF2, address:77.90.31.16:4747): protocol: rpc: dial tcp 77.90.31.16:4747: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042966233 (355h50m3.969260923s) [noise=true , address=77.90.31.19:4856 , piece_id=CR7AX6HZZGBQM5QATGLDLHQGUPZC4KW6H5CIBB4LFRTNL2WOLQFQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:1MntjJtcCDSRozKpesp6S2ZjYrMhxiQheDiPMPSCnavWBBGYt3, address:77.90.31.19:4856): protocol: rpc: dial tcp 77.90.31.19:4856: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043090813 (355h50m3.969385503s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: write tcp 159.223.143.39:47434->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043180860 (355h50m3.969475548s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12Ttdx6LHmuAt8JVC7arCDgWjm9eLadjeaEw17F12btoXuSiq9t, address:77.90.31.11:4583): protocol: write tcp 159.223.143.39:43298->77.90.31.11:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043193162 (355h50m3.969487876s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12LuFBTJg1aNmZoEdiQTtvfnfh5qnWVZFtqFX93L4fNfiYTYqJ6, address:77.90.31.18:4818): protocol: rpc: dial tcp 77.90.31.18:4818: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043283221 (355h50m3.969577895s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:126Sou5pnMm2eE2FJpv6WYX2VSqJvZZf9rFFJD6hFYDetRD28zu, address:77.90.31.12:4603): protocol: rpc: dial tcp 77.90.31.12:4603: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043307928 (355h50m3.969602654s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043494443 (355h50m3.969789132s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:168Tw7W69Tq9opDKhJ4cM8PGnzpxp9E9SxKbdVxJeF83pETqGx, address:77.90.31.17:4804): protocol: rpc: dial tcp 77.90.31.17:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043560259 (355h50m3.969854956s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: write tcp 159.223.143.39:37628->77.90.31.13:4632: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043598718 (355h50m3.969893411s) [seg_pos={0 0}, upload=284991]: Done waiting for segment. successful: 8 optimal: 14 errs: <nil> Jul 8 05:01:37.086050110 (355h50m4.012344788s) [upload=284991]: Done uploading segments... Jul 8 05:01:37.086091388 (355h50m4.012386065s) [upload=284991]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 8) Duration: 425.578422ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................................XXXX.......................] duration: 379.729258ms FAIL (2025-07-08 05:16:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== GPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674108143 (355h49m20.600402838s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674596780 (355h49m20.600891471s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675604361 (355h49m20.601899058s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675768580 (355h49m20.602063277s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676120119 (355h49m20.60241481s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676215473 (355h49m20.602510169s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676623023 (355h49m20.602917716s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676647235 (355h49m20.602941929s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676696624 (355h49m20.602991321s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677177615 (355h49m20.603472312s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677707610 (355h49m20.604002306s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678250080 (355h49m20.604544775s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678907662 (355h49m20.605202358s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679453605 (355h49m20.6057483s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679618853 (355h49m20.605913551s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.771112310 (355h49m20.697407s) [upload=284997]: Next returned nil segment Jul 8 05:00:53.771144706 (355h49m20.697439396s) [upload=284997]: Waiting for error group managing segments... Jul 8 05:00:53.780217385 (355h49m20.70651207s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.780770161 (355h49m20.70706484s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781213801 (355h49m20.707508488s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781712476 (355h49m20.708007155s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.782192975 (355h49m20.708487647s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:01:23.612230509 (355h49m50.538525197s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612301272 (355h49m50.538595951s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 159.223.143.39:45850->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612349516 (355h49m50.538644191s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 159.223.143.39:38418->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612454902 (355h49m50.538749592s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612531645 (355h49m50.538826322s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 159.223.143.39:39432->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612552075 (355h49m50.538846753s) [noise=true , address=45.91.116.11:4118 , piece_id=KFPPGPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 159.223.143.39:33394->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612586657 (355h49m50.538881334s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 159.223.143.39:49070->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612604791 (355h49m50.538899468s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612642895 (355h49m50.538937572s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612651542 (355h49m50.538946218s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 159.223.143.39:47692->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612680902 (355h49m50.538975589s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612713168 (355h49m50.539007856s) [seg_pos={0 0}, upload=284997]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 8 05:01:23.648975097 (355h49m50.575269777s) [upload=284997]: Done uploading segments... Jul 8 05:01:23.649010451 (355h49m50.575305138s) [upload=284997]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 379.78548ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.....................................XXX....................] duration: 432.106704ms FAIL (2025-07-08 05:16:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 8) ======LOGS====== pload=284991]: Uploading piece... Jul 8 05:00:37.137063656 (355h49m4.063358349s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.137984339 (355h49m4.064279032s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138034396 (355h49m4.064329092s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138836689 (355h49m4.065131384s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138884136 (355h49m4.065178829s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139428483 (355h49m4.065723178s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139436408 (355h49m4.0657311s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139788656 (355h49m4.066083348s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139920627 (355h49m4.066215325s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140071882 (355h49m4.066366572s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140353654 (355h49m4.06664834s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140415319 (355h49m4.066710015s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140681286 (355h49m4.066975982s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140973150 (355h49m4.067267847s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.144728804 (355h49m4.071023482s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.249883590 (355h49m4.176178265s) [upload=284991]: Next returned nil segment Jul 8 05:00:37.249902715 (355h49m4.176197412s) [upload=284991]: Waiting for error group managing segments... Jul 8 05:00:37.260212837 (355h49m4.186507526s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260803813 (355h49m4.187098502s) [noise=true , address=104.36.82.45:4915 , piece_id=G6IVO3DOQGN5GQKW67NX2ZKBOVTQEBA2PUNAGFDIKUI6OLPGMZHQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260834568 (355h49m4.187129264s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261367510 (355h49m4.187662188s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261455746 (355h49m4.187750422s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261903476 (355h49m4.188198155s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.278055422 (355h49m4.204350095s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.300910056 (355h49m4.227204736s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:01:37.042774308 (355h50m3.969069003s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:19LE2Lq4e6HojnMp9xQ3ySLkRe5PemXTwiQZThN5jT46bQhN6K, address:77.90.31.19:4855): protocol: rpc: dial tcp 77.90.31.19:4855: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042887419 (355h50m3.969182106s) [noise=true , address=77.90.31.16:4747 , piece_id=HV6IOGCMOHIUEYTGJMQ62V4UUNJ2MQD37SFF5Y23X2S66YYQSN3Q, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12ERdBm3XPw4s7JHeMWsVniwcqnfMtP4ZX8sBVjri6dzZtq8AF2, address:77.90.31.16:4747): protocol: rpc: dial tcp 77.90.31.16:4747: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042966233 (355h50m3.969260923s) [noise=true , address=77.90.31.19:4856 , piece_id=CR7AX6HZZGBQM5QATGLDLHQGUPZC4KW6H5CIBB4LFRTNL2WOLQFQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:1MntjJtcCDSRozKpesp6S2ZjYrMhxiQheDiPMPSCnavWBBGYt3, address:77.90.31.19:4856): protocol: rpc: dial tcp 77.90.31.19:4856: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043090813 (355h50m3.969385503s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: write tcp 159.223.143.39:47434->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043180860 (355h50m3.969475548s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12Ttdx6LHmuAt8JVC7arCDgWjm9eLadjeaEw17F12btoXuSiq9t, address:77.90.31.11:4583): protocol: write tcp 159.223.143.39:43298->77.90.31.11:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043193162 (355h50m3.969487876s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12LuFBTJg1aNmZoEdiQTtvfnfh5qnWVZFtqFX93L4fNfiYTYqJ6, address:77.90.31.18:4818): protocol: rpc: dial tcp 77.90.31.18:4818: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043283221 (355h50m3.969577895s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:126Sou5pnMm2eE2FJpv6WYX2VSqJvZZf9rFFJD6hFYDetRD28zu, address:77.90.31.12:4603): protocol: rpc: dial tcp 77.90.31.12:4603: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043307928 (355h50m3.969602654s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043494443 (355h50m3.969789132s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:168Tw7W69Tq9opDKhJ4cM8PGnzpxp9E9SxKbdVxJeF83pETqGx, address:77.90.31.17:4804): protocol: rpc: dial tcp 77.90.31.17:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043560259 (355h50m3.969854956s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: write tcp 159.223.143.39:37628->77.90.31.13:4632: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043598718 (355h50m3.969893411s) [seg_pos={0 0}, upload=284991]: Done waiting for segment. successful: 8 optimal: 14 errs: <nil> Jul 8 05:01:37.086050110 (355h50m4.012344788s) [upload=284991]: Done uploading segments... Jul 8 05:01:37.086091388 (355h50m4.012386065s) [upload=284991]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 8) Duration: 432.139982ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................................XXXX......................] duration: 499.240495ms FAIL (2025-07-08 05:15:53 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== GPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674108143 (355h49m20.600402838s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674596780 (355h49m20.600891471s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675604361 (355h49m20.601899058s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675768580 (355h49m20.602063277s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676120119 (355h49m20.60241481s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676215473 (355h49m20.602510169s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676623023 (355h49m20.602917716s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676647235 (355h49m20.602941929s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676696624 (355h49m20.602991321s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677177615 (355h49m20.603472312s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677707610 (355h49m20.604002306s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678250080 (355h49m20.604544775s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678907662 (355h49m20.605202358s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679453605 (355h49m20.6057483s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679618853 (355h49m20.605913551s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.771112310 (355h49m20.697407s) [upload=284997]: Next returned nil segment Jul 8 05:00:53.771144706 (355h49m20.697439396s) [upload=284997]: Waiting for error group managing segments... Jul 8 05:00:53.780217385 (355h49m20.70651207s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.780770161 (355h49m20.70706484s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781213801 (355h49m20.707508488s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781712476 (355h49m20.708007155s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.782192975 (355h49m20.708487647s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:01:23.612230509 (355h49m50.538525197s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612301272 (355h49m50.538595951s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 159.223.143.39:45850->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612349516 (355h49m50.538644191s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 159.223.143.39:38418->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612454902 (355h49m50.538749592s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612531645 (355h49m50.538826322s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 159.223.143.39:39432->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612552075 (355h49m50.538846753s) [noise=true , address=45.91.116.11:4118 , piece_id=KFPPGPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 159.223.143.39:33394->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612586657 (355h49m50.538881334s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 159.223.143.39:49070->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612604791 (355h49m50.538899468s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612642895 (355h49m50.538937572s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612651542 (355h49m50.538946218s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 159.223.143.39:47692->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612680902 (355h49m50.538975589s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612713168 (355h49m50.539007856s) [seg_pos={0 0}, upload=284997]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 8 05:01:23.648975097 (355h49m50.575269777s) [upload=284997]: Done uploading segments... Jul 8 05:01:23.649010451 (355h49m50.575305138s) [upload=284997]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 499.269101ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [......................................XXX...................] duration: 563.576264ms FAIL (2025-07-08 05:15:53 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 8) ======LOGS====== pload=284991]: Uploading piece... Jul 8 05:00:37.137063656 (355h49m4.063358349s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.137984339 (355h49m4.064279032s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138034396 (355h49m4.064329092s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138836689 (355h49m4.065131384s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138884136 (355h49m4.065178829s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139428483 (355h49m4.065723178s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139436408 (355h49m4.0657311s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139788656 (355h49m4.066083348s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139920627 (355h49m4.066215325s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140071882 (355h49m4.066366572s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140353654 (355h49m4.06664834s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140415319 (355h49m4.066710015s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140681286 (355h49m4.066975982s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140973150 (355h49m4.067267847s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.144728804 (355h49m4.071023482s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.249883590 (355h49m4.176178265s) [upload=284991]: Next returned nil segment Jul 8 05:00:37.249902715 (355h49m4.176197412s) [upload=284991]: Waiting for error group managing segments... Jul 8 05:00:37.260212837 (355h49m4.186507526s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260803813 (355h49m4.187098502s) [noise=true , address=104.36.82.45:4915 , piece_id=G6IVO3DOQGN5GQKW67NX2ZKBOVTQEBA2PUNAGFDIKUI6OLPGMZHQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260834568 (355h49m4.187129264s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261367510 (355h49m4.187662188s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261455746 (355h49m4.187750422s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261903476 (355h49m4.188198155s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.278055422 (355h49m4.204350095s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.300910056 (355h49m4.227204736s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:01:37.042774308 (355h50m3.969069003s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:19LE2Lq4e6HojnMp9xQ3ySLkRe5PemXTwiQZThN5jT46bQhN6K, address:77.90.31.19:4855): protocol: rpc: dial tcp 77.90.31.19:4855: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042887419 (355h50m3.969182106s) [noise=true , address=77.90.31.16:4747 , piece_id=HV6IOGCMOHIUEYTGJMQ62V4UUNJ2MQD37SFF5Y23X2S66YYQSN3Q, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12ERdBm3XPw4s7JHeMWsVniwcqnfMtP4ZX8sBVjri6dzZtq8AF2, address:77.90.31.16:4747): protocol: rpc: dial tcp 77.90.31.16:4747: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042966233 (355h50m3.969260923s) [noise=true , address=77.90.31.19:4856 , piece_id=CR7AX6HZZGBQM5QATGLDLHQGUPZC4KW6H5CIBB4LFRTNL2WOLQFQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:1MntjJtcCDSRozKpesp6S2ZjYrMhxiQheDiPMPSCnavWBBGYt3, address:77.90.31.19:4856): protocol: rpc: dial tcp 77.90.31.19:4856: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043090813 (355h50m3.969385503s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: write tcp 159.223.143.39:47434->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043180860 (355h50m3.969475548s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12Ttdx6LHmuAt8JVC7arCDgWjm9eLadjeaEw17F12btoXuSiq9t, address:77.90.31.11:4583): protocol: write tcp 159.223.143.39:43298->77.90.31.11:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043193162 (355h50m3.969487876s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12LuFBTJg1aNmZoEdiQTtvfnfh5qnWVZFtqFX93L4fNfiYTYqJ6, address:77.90.31.18:4818): protocol: rpc: dial tcp 77.90.31.18:4818: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043283221 (355h50m3.969577895s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:126Sou5pnMm2eE2FJpv6WYX2VSqJvZZf9rFFJD6hFYDetRD28zu, address:77.90.31.12:4603): protocol: rpc: dial tcp 77.90.31.12:4603: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043307928 (355h50m3.969602654s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043494443 (355h50m3.969789132s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:168Tw7W69Tq9opDKhJ4cM8PGnzpxp9E9SxKbdVxJeF83pETqGx, address:77.90.31.17:4804): protocol: rpc: dial tcp 77.90.31.17:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043560259 (355h50m3.969854956s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: write tcp 159.223.143.39:37628->77.90.31.13:4632: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043598718 (355h50m3.969893411s) [seg_pos={0 0}, upload=284991]: Done waiting for segment. successful: 8 optimal: 14 errs: <nil> Jul 8 05:01:37.086050110 (355h50m4.012344788s) [upload=284991]: Done uploading segments... Jul 8 05:01:37.086091388 (355h50m4.012386065s) [upload=284991]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 8) Duration: 563.648097ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.......................................XXX..................] duration: 462.275832ms FAIL (2025-07-08 05:15:19 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 8) ======LOGS====== pload=284991]: Uploading piece... Jul 8 05:00:37.137063656 (355h49m4.063358349s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.137984339 (355h49m4.064279032s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138034396 (355h49m4.064329092s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138836689 (355h49m4.065131384s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138884136 (355h49m4.065178829s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139428483 (355h49m4.065723178s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139436408 (355h49m4.0657311s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139788656 (355h49m4.066083348s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139920627 (355h49m4.066215325s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140071882 (355h49m4.066366572s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140353654 (355h49m4.06664834s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140415319 (355h49m4.066710015s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140681286 (355h49m4.066975982s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140973150 (355h49m4.067267847s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.144728804 (355h49m4.071023482s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.249883590 (355h49m4.176178265s) [upload=284991]: Next returned nil segment Jul 8 05:00:37.249902715 (355h49m4.176197412s) [upload=284991]: Waiting for error group managing segments... Jul 8 05:00:37.260212837 (355h49m4.186507526s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260803813 (355h49m4.187098502s) [noise=true , address=104.36.82.45:4915 , piece_id=G6IVO3DOQGN5GQKW67NX2ZKBOVTQEBA2PUNAGFDIKUI6OLPGMZHQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260834568 (355h49m4.187129264s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261367510 (355h49m4.187662188s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261455746 (355h49m4.187750422s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261903476 (355h49m4.188198155s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.278055422 (355h49m4.204350095s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.300910056 (355h49m4.227204736s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:01:37.042774308 (355h50m3.969069003s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:19LE2Lq4e6HojnMp9xQ3ySLkRe5PemXTwiQZThN5jT46bQhN6K, address:77.90.31.19:4855): protocol: rpc: dial tcp 77.90.31.19:4855: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042887419 (355h50m3.969182106s) [noise=true , address=77.90.31.16:4747 , piece_id=HV6IOGCMOHIUEYTGJMQ62V4UUNJ2MQD37SFF5Y23X2S66YYQSN3Q, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12ERdBm3XPw4s7JHeMWsVniwcqnfMtP4ZX8sBVjri6dzZtq8AF2, address:77.90.31.16:4747): protocol: rpc: dial tcp 77.90.31.16:4747: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042966233 (355h50m3.969260923s) [noise=true , address=77.90.31.19:4856 , piece_id=CR7AX6HZZGBQM5QATGLDLHQGUPZC4KW6H5CIBB4LFRTNL2WOLQFQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:1MntjJtcCDSRozKpesp6S2ZjYrMhxiQheDiPMPSCnavWBBGYt3, address:77.90.31.19:4856): protocol: rpc: dial tcp 77.90.31.19:4856: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043090813 (355h50m3.969385503s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: write tcp 159.223.143.39:47434->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043180860 (355h50m3.969475548s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12Ttdx6LHmuAt8JVC7arCDgWjm9eLadjeaEw17F12btoXuSiq9t, address:77.90.31.11:4583): protocol: write tcp 159.223.143.39:43298->77.90.31.11:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043193162 (355h50m3.969487876s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12LuFBTJg1aNmZoEdiQTtvfnfh5qnWVZFtqFX93L4fNfiYTYqJ6, address:77.90.31.18:4818): protocol: rpc: dial tcp 77.90.31.18:4818: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043283221 (355h50m3.969577895s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:126Sou5pnMm2eE2FJpv6WYX2VSqJvZZf9rFFJD6hFYDetRD28zu, address:77.90.31.12:4603): protocol: rpc: dial tcp 77.90.31.12:4603: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043307928 (355h50m3.969602654s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043494443 (355h50m3.969789132s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:168Tw7W69Tq9opDKhJ4cM8PGnzpxp9E9SxKbdVxJeF83pETqGx, address:77.90.31.17:4804): protocol: rpc: dial tcp 77.90.31.17:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043560259 (355h50m3.969854956s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: write tcp 159.223.143.39:37628->77.90.31.13:4632: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043598718 (355h50m3.969893411s) [seg_pos={0 0}, upload=284991]: Done waiting for segment. successful: 8 optimal: 14 errs: <nil> Jul 8 05:01:37.086050110 (355h50m4.012344788s) [upload=284991]: Done uploading segments... Jul 8 05:01:37.086091388 (355h50m4.012386065s) [upload=284991]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 8) Duration: 462.359366ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................................XXXX.....................] duration: 400.467506ms FAIL (2025-07-08 05:14:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== GPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674108143 (355h49m20.600402838s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674596780 (355h49m20.600891471s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675604361 (355h49m20.601899058s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675768580 (355h49m20.602063277s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676120119 (355h49m20.60241481s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676215473 (355h49m20.602510169s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676623023 (355h49m20.602917716s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676647235 (355h49m20.602941929s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676696624 (355h49m20.602991321s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677177615 (355h49m20.603472312s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677707610 (355h49m20.604002306s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678250080 (355h49m20.604544775s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678907662 (355h49m20.605202358s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679453605 (355h49m20.6057483s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679618853 (355h49m20.605913551s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.771112310 (355h49m20.697407s) [upload=284997]: Next returned nil segment Jul 8 05:00:53.771144706 (355h49m20.697439396s) [upload=284997]: Waiting for error group managing segments... Jul 8 05:00:53.780217385 (355h49m20.70651207s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.780770161 (355h49m20.70706484s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781213801 (355h49m20.707508488s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781712476 (355h49m20.708007155s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.782192975 (355h49m20.708487647s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:01:23.612230509 (355h49m50.538525197s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612301272 (355h49m50.538595951s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 159.223.143.39:45850->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612349516 (355h49m50.538644191s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 159.223.143.39:38418->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612454902 (355h49m50.538749592s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612531645 (355h49m50.538826322s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 159.223.143.39:39432->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612552075 (355h49m50.538846753s) [noise=true , address=45.91.116.11:4118 , piece_id=KFPPGPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 159.223.143.39:33394->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612586657 (355h49m50.538881334s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 159.223.143.39:49070->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612604791 (355h49m50.538899468s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612642895 (355h49m50.538937572s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612651542 (355h49m50.538946218s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 159.223.143.39:47692->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612680902 (355h49m50.538975589s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612713168 (355h49m50.539007856s) [seg_pos={0 0}, upload=284997]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 8 05:01:23.648975097 (355h49m50.575269777s) [upload=284997]: Done uploading segments... Jul 8 05:01:23.649010451 (355h49m50.575305138s) [upload=284997]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 400.502155ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [........................................XXX.................] duration: 554.870194ms FAIL (2025-07-08 05:14:16 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 8) ======LOGS====== pload=284991]: Uploading piece... Jul 8 05:00:37.137063656 (355h49m4.063358349s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.137984339 (355h49m4.064279032s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138034396 (355h49m4.064329092s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138836689 (355h49m4.065131384s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138884136 (355h49m4.065178829s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139428483 (355h49m4.065723178s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139436408 (355h49m4.0657311s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139788656 (355h49m4.066083348s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139920627 (355h49m4.066215325s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140071882 (355h49m4.066366572s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140353654 (355h49m4.06664834s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140415319 (355h49m4.066710015s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140681286 (355h49m4.066975982s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140973150 (355h49m4.067267847s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.144728804 (355h49m4.071023482s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.249883590 (355h49m4.176178265s) [upload=284991]: Next returned nil segment Jul 8 05:00:37.249902715 (355h49m4.176197412s) [upload=284991]: Waiting for error group managing segments... Jul 8 05:00:37.260212837 (355h49m4.186507526s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260803813 (355h49m4.187098502s) [noise=true , address=104.36.82.45:4915 , piece_id=G6IVO3DOQGN5GQKW67NX2ZKBOVTQEBA2PUNAGFDIKUI6OLPGMZHQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260834568 (355h49m4.187129264s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261367510 (355h49m4.187662188s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261455746 (355h49m4.187750422s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261903476 (355h49m4.188198155s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.278055422 (355h49m4.204350095s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.300910056 (355h49m4.227204736s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:01:37.042774308 (355h50m3.969069003s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:19LE2Lq4e6HojnMp9xQ3ySLkRe5PemXTwiQZThN5jT46bQhN6K, address:77.90.31.19:4855): protocol: rpc: dial tcp 77.90.31.19:4855: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042887419 (355h50m3.969182106s) [noise=true , address=77.90.31.16:4747 , piece_id=HV6IOGCMOHIUEYTGJMQ62V4UUNJ2MQD37SFF5Y23X2S66YYQSN3Q, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12ERdBm3XPw4s7JHeMWsVniwcqnfMtP4ZX8sBVjri6dzZtq8AF2, address:77.90.31.16:4747): protocol: rpc: dial tcp 77.90.31.16:4747: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042966233 (355h50m3.969260923s) [noise=true , address=77.90.31.19:4856 , piece_id=CR7AX6HZZGBQM5QATGLDLHQGUPZC4KW6H5CIBB4LFRTNL2WOLQFQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:1MntjJtcCDSRozKpesp6S2ZjYrMhxiQheDiPMPSCnavWBBGYt3, address:77.90.31.19:4856): protocol: rpc: dial tcp 77.90.31.19:4856: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043090813 (355h50m3.969385503s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: write tcp 159.223.143.39:47434->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043180860 (355h50m3.969475548s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12Ttdx6LHmuAt8JVC7arCDgWjm9eLadjeaEw17F12btoXuSiq9t, address:77.90.31.11:4583): protocol: write tcp 159.223.143.39:43298->77.90.31.11:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043193162 (355h50m3.969487876s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12LuFBTJg1aNmZoEdiQTtvfnfh5qnWVZFtqFX93L4fNfiYTYqJ6, address:77.90.31.18:4818): protocol: rpc: dial tcp 77.90.31.18:4818: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043283221 (355h50m3.969577895s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:126Sou5pnMm2eE2FJpv6WYX2VSqJvZZf9rFFJD6hFYDetRD28zu, address:77.90.31.12:4603): protocol: rpc: dial tcp 77.90.31.12:4603: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043307928 (355h50m3.969602654s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043494443 (355h50m3.969789132s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:168Tw7W69Tq9opDKhJ4cM8PGnzpxp9E9SxKbdVxJeF83pETqGx, address:77.90.31.17:4804): protocol: rpc: dial tcp 77.90.31.17:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043560259 (355h50m3.969854956s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: write tcp 159.223.143.39:37628->77.90.31.13:4632: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043598718 (355h50m3.969893411s) [seg_pos={0 0}, upload=284991]: Done waiting for segment. successful: 8 optimal: 14 errs: <nil> Jul 8 05:01:37.086050110 (355h50m4.012344788s) [upload=284991]: Done uploading segments... Jul 8 05:01:37.086091388 (355h50m4.012386065s) [upload=284991]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 8) Duration: 554.904551ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................................XXXX....................] duration: 473.429069ms FAIL (2025-07-08 05:13:47 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== GPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674108143 (355h49m20.600402838s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674596780 (355h49m20.600891471s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675604361 (355h49m20.601899058s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675768580 (355h49m20.602063277s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676120119 (355h49m20.60241481s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676215473 (355h49m20.602510169s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676623023 (355h49m20.602917716s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676647235 (355h49m20.602941929s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676696624 (355h49m20.602991321s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677177615 (355h49m20.603472312s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677707610 (355h49m20.604002306s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678250080 (355h49m20.604544775s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678907662 (355h49m20.605202358s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679453605 (355h49m20.6057483s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679618853 (355h49m20.605913551s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.771112310 (355h49m20.697407s) [upload=284997]: Next returned nil segment Jul 8 05:00:53.771144706 (355h49m20.697439396s) [upload=284997]: Waiting for error group managing segments... Jul 8 05:00:53.780217385 (355h49m20.70651207s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.780770161 (355h49m20.70706484s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781213801 (355h49m20.707508488s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781712476 (355h49m20.708007155s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.782192975 (355h49m20.708487647s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:01:23.612230509 (355h49m50.538525197s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612301272 (355h49m50.538595951s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 159.223.143.39:45850->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612349516 (355h49m50.538644191s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 159.223.143.39:38418->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612454902 (355h49m50.538749592s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612531645 (355h49m50.538826322s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 159.223.143.39:39432->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612552075 (355h49m50.538846753s) [noise=true , address=45.91.116.11:4118 , piece_id=KFPPGPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 159.223.143.39:33394->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612586657 (355h49m50.538881334s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 159.223.143.39:49070->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612604791 (355h49m50.538899468s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612642895 (355h49m50.538937572s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612651542 (355h49m50.538946218s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 159.223.143.39:47692->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612680902 (355h49m50.538975589s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612713168 (355h49m50.539007856s) [seg_pos={0 0}, upload=284997]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 8 05:01:23.648975097 (355h49m50.575269777s) [upload=284997]: Done uploading segments... Jul 8 05:01:23.649010451 (355h49m50.575305138s) [upload=284997]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 473.459379ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.........................................XXX................] duration: 480.488785ms FAIL (2025-07-08 05:13:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 8) ======LOGS====== pload=284991]: Uploading piece... Jul 8 05:00:37.137063656 (355h49m4.063358349s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.137984339 (355h49m4.064279032s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138034396 (355h49m4.064329092s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138836689 (355h49m4.065131384s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138884136 (355h49m4.065178829s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139428483 (355h49m4.065723178s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139436408 (355h49m4.0657311s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139788656 (355h49m4.066083348s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139920627 (355h49m4.066215325s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140071882 (355h49m4.066366572s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140353654 (355h49m4.06664834s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140415319 (355h49m4.066710015s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140681286 (355h49m4.066975982s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140973150 (355h49m4.067267847s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.144728804 (355h49m4.071023482s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.249883590 (355h49m4.176178265s) [upload=284991]: Next returned nil segment Jul 8 05:00:37.249902715 (355h49m4.176197412s) [upload=284991]: Waiting for error group managing segments... Jul 8 05:00:37.260212837 (355h49m4.186507526s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260803813 (355h49m4.187098502s) [noise=true , address=104.36.82.45:4915 , piece_id=G6IVO3DOQGN5GQKW67NX2ZKBOVTQEBA2PUNAGFDIKUI6OLPGMZHQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260834568 (355h49m4.187129264s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261367510 (355h49m4.187662188s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261455746 (355h49m4.187750422s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261903476 (355h49m4.188198155s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.278055422 (355h49m4.204350095s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.300910056 (355h49m4.227204736s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:01:37.042774308 (355h50m3.969069003s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:19LE2Lq4e6HojnMp9xQ3ySLkRe5PemXTwiQZThN5jT46bQhN6K, address:77.90.31.19:4855): protocol: rpc: dial tcp 77.90.31.19:4855: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042887419 (355h50m3.969182106s) [noise=true , address=77.90.31.16:4747 , piece_id=HV6IOGCMOHIUEYTGJMQ62V4UUNJ2MQD37SFF5Y23X2S66YYQSN3Q, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12ERdBm3XPw4s7JHeMWsVniwcqnfMtP4ZX8sBVjri6dzZtq8AF2, address:77.90.31.16:4747): protocol: rpc: dial tcp 77.90.31.16:4747: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042966233 (355h50m3.969260923s) [noise=true , address=77.90.31.19:4856 , piece_id=CR7AX6HZZGBQM5QATGLDLHQGUPZC4KW6H5CIBB4LFRTNL2WOLQFQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:1MntjJtcCDSRozKpesp6S2ZjYrMhxiQheDiPMPSCnavWBBGYt3, address:77.90.31.19:4856): protocol: rpc: dial tcp 77.90.31.19:4856: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043090813 (355h50m3.969385503s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: write tcp 159.223.143.39:47434->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043180860 (355h50m3.969475548s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12Ttdx6LHmuAt8JVC7arCDgWjm9eLadjeaEw17F12btoXuSiq9t, address:77.90.31.11:4583): protocol: write tcp 159.223.143.39:43298->77.90.31.11:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043193162 (355h50m3.969487876s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12LuFBTJg1aNmZoEdiQTtvfnfh5qnWVZFtqFX93L4fNfiYTYqJ6, address:77.90.31.18:4818): protocol: rpc: dial tcp 77.90.31.18:4818: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043283221 (355h50m3.969577895s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:126Sou5pnMm2eE2FJpv6WYX2VSqJvZZf9rFFJD6hFYDetRD28zu, address:77.90.31.12:4603): protocol: rpc: dial tcp 77.90.31.12:4603: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043307928 (355h50m3.969602654s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043494443 (355h50m3.969789132s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:168Tw7W69Tq9opDKhJ4cM8PGnzpxp9E9SxKbdVxJeF83pETqGx, address:77.90.31.17:4804): protocol: rpc: dial tcp 77.90.31.17:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043560259 (355h50m3.969854956s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: write tcp 159.223.143.39:37628->77.90.31.13:4632: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043598718 (355h50m3.969893411s) [seg_pos={0 0}, upload=284991]: Done waiting for segment. successful: 8 optimal: 14 errs: <nil> Jul 8 05:01:37.086050110 (355h50m4.012344788s) [upload=284991]: Done uploading segments... Jul 8 05:01:37.086091388 (355h50m4.012386065s) [upload=284991]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 8) Duration: 480.515393ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................................XXXX...................] duration: 465.387685ms FAIL (2025-07-08 05:12:46 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== GPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674108143 (355h49m20.600402838s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674596780 (355h49m20.600891471s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675604361 (355h49m20.601899058s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675768580 (355h49m20.602063277s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676120119 (355h49m20.60241481s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676215473 (355h49m20.602510169s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676623023 (355h49m20.602917716s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676647235 (355h49m20.602941929s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676696624 (355h49m20.602991321s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677177615 (355h49m20.603472312s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677707610 (355h49m20.604002306s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678250080 (355h49m20.604544775s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678907662 (355h49m20.605202358s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679453605 (355h49m20.6057483s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679618853 (355h49m20.605913551s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.771112310 (355h49m20.697407s) [upload=284997]: Next returned nil segment Jul 8 05:00:53.771144706 (355h49m20.697439396s) [upload=284997]: Waiting for error group managing segments... Jul 8 05:00:53.780217385 (355h49m20.70651207s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.780770161 (355h49m20.70706484s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781213801 (355h49m20.707508488s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781712476 (355h49m20.708007155s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.782192975 (355h49m20.708487647s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:01:23.612230509 (355h49m50.538525197s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612301272 (355h49m50.538595951s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 159.223.143.39:45850->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612349516 (355h49m50.538644191s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 159.223.143.39:38418->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612454902 (355h49m50.538749592s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612531645 (355h49m50.538826322s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 159.223.143.39:39432->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612552075 (355h49m50.538846753s) [noise=true , address=45.91.116.11:4118 , piece_id=KFPPGPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 159.223.143.39:33394->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612586657 (355h49m50.538881334s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 159.223.143.39:49070->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612604791 (355h49m50.538899468s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612642895 (355h49m50.538937572s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612651542 (355h49m50.538946218s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 159.223.143.39:47692->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612680902 (355h49m50.538975589s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612713168 (355h49m50.539007856s) [seg_pos={0 0}, upload=284997]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 8 05:01:23.648975097 (355h49m50.575269777s) [upload=284997]: Done uploading segments... Jul 8 05:01:23.649010451 (355h49m50.575305138s) [upload=284997]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 465.419344ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [..........................................XXX...............] duration: 457.850424ms FAIL (2025-07-08 05:12:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 8) ======LOGS====== pload=284991]: Uploading piece... Jul 8 05:00:37.137063656 (355h49m4.063358349s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.137984339 (355h49m4.064279032s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138034396 (355h49m4.064329092s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138836689 (355h49m4.065131384s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138884136 (355h49m4.065178829s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139428483 (355h49m4.065723178s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139436408 (355h49m4.0657311s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139788656 (355h49m4.066083348s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139920627 (355h49m4.066215325s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140071882 (355h49m4.066366572s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140353654 (355h49m4.06664834s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140415319 (355h49m4.066710015s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140681286 (355h49m4.066975982s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140973150 (355h49m4.067267847s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.144728804 (355h49m4.071023482s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.249883590 (355h49m4.176178265s) [upload=284991]: Next returned nil segment Jul 8 05:00:37.249902715 (355h49m4.176197412s) [upload=284991]: Waiting for error group managing segments... Jul 8 05:00:37.260212837 (355h49m4.186507526s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260803813 (355h49m4.187098502s) [noise=true , address=104.36.82.45:4915 , piece_id=G6IVO3DOQGN5GQKW67NX2ZKBOVTQEBA2PUNAGFDIKUI6OLPGMZHQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260834568 (355h49m4.187129264s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261367510 (355h49m4.187662188s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261455746 (355h49m4.187750422s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261903476 (355h49m4.188198155s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.278055422 (355h49m4.204350095s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.300910056 (355h49m4.227204736s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:01:37.042774308 (355h50m3.969069003s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:19LE2Lq4e6HojnMp9xQ3ySLkRe5PemXTwiQZThN5jT46bQhN6K, address:77.90.31.19:4855): protocol: rpc: dial tcp 77.90.31.19:4855: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042887419 (355h50m3.969182106s) [noise=true , address=77.90.31.16:4747 , piece_id=HV6IOGCMOHIUEYTGJMQ62V4UUNJ2MQD37SFF5Y23X2S66YYQSN3Q, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12ERdBm3XPw4s7JHeMWsVniwcqnfMtP4ZX8sBVjri6dzZtq8AF2, address:77.90.31.16:4747): protocol: rpc: dial tcp 77.90.31.16:4747: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042966233 (355h50m3.969260923s) [noise=true , address=77.90.31.19:4856 , piece_id=CR7AX6HZZGBQM5QATGLDLHQGUPZC4KW6H5CIBB4LFRTNL2WOLQFQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:1MntjJtcCDSRozKpesp6S2ZjYrMhxiQheDiPMPSCnavWBBGYt3, address:77.90.31.19:4856): protocol: rpc: dial tcp 77.90.31.19:4856: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043090813 (355h50m3.969385503s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: write tcp 159.223.143.39:47434->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043180860 (355h50m3.969475548s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12Ttdx6LHmuAt8JVC7arCDgWjm9eLadjeaEw17F12btoXuSiq9t, address:77.90.31.11:4583): protocol: write tcp 159.223.143.39:43298->77.90.31.11:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043193162 (355h50m3.969487876s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12LuFBTJg1aNmZoEdiQTtvfnfh5qnWVZFtqFX93L4fNfiYTYqJ6, address:77.90.31.18:4818): protocol: rpc: dial tcp 77.90.31.18:4818: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043283221 (355h50m3.969577895s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:126Sou5pnMm2eE2FJpv6WYX2VSqJvZZf9rFFJD6hFYDetRD28zu, address:77.90.31.12:4603): protocol: rpc: dial tcp 77.90.31.12:4603: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043307928 (355h50m3.969602654s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043494443 (355h50m3.969789132s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:168Tw7W69Tq9opDKhJ4cM8PGnzpxp9E9SxKbdVxJeF83pETqGx, address:77.90.31.17:4804): protocol: rpc: dial tcp 77.90.31.17:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043560259 (355h50m3.969854956s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: write tcp 159.223.143.39:37628->77.90.31.13:4632: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043598718 (355h50m3.969893411s) [seg_pos={0 0}, upload=284991]: Done waiting for segment. successful: 8 optimal: 14 errs: <nil> Jul 8 05:01:37.086050110 (355h50m4.012344788s) [upload=284991]: Done uploading segments... Jul 8 05:01:37.086091388 (355h50m4.012386065s) [upload=284991]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 8) Duration: 457.88515ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................................XXXX..................] duration: 462.877202ms FAIL (2025-07-08 05:12:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== GPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674108143 (355h49m20.600402838s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674596780 (355h49m20.600891471s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675604361 (355h49m20.601899058s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675768580 (355h49m20.602063277s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676120119 (355h49m20.60241481s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676215473 (355h49m20.602510169s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676623023 (355h49m20.602917716s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676647235 (355h49m20.602941929s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676696624 (355h49m20.602991321s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677177615 (355h49m20.603472312s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677707610 (355h49m20.604002306s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678250080 (355h49m20.604544775s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678907662 (355h49m20.605202358s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679453605 (355h49m20.6057483s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679618853 (355h49m20.605913551s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.771112310 (355h49m20.697407s) [upload=284997]: Next returned nil segment Jul 8 05:00:53.771144706 (355h49m20.697439396s) [upload=284997]: Waiting for error group managing segments... Jul 8 05:00:53.780217385 (355h49m20.70651207s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.780770161 (355h49m20.70706484s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781213801 (355h49m20.707508488s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781712476 (355h49m20.708007155s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.782192975 (355h49m20.708487647s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:01:23.612230509 (355h49m50.538525197s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612301272 (355h49m50.538595951s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 159.223.143.39:45850->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612349516 (355h49m50.538644191s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 159.223.143.39:38418->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612454902 (355h49m50.538749592s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612531645 (355h49m50.538826322s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 159.223.143.39:39432->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612552075 (355h49m50.538846753s) [noise=true , address=45.91.116.11:4118 , piece_id=KFPPGPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 159.223.143.39:33394->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612586657 (355h49m50.538881334s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 159.223.143.39:49070->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612604791 (355h49m50.538899468s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612642895 (355h49m50.538937572s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612651542 (355h49m50.538946218s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 159.223.143.39:47692->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612680902 (355h49m50.538975589s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612713168 (355h49m50.539007856s) [seg_pos={0 0}, upload=284997]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 8 05:01:23.648975097 (355h49m50.575269777s) [upload=284997]: Done uploading segments... Jul 8 05:01:23.649010451 (355h49m50.575305138s) [upload=284997]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 462.905758ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................................XXXX.................] duration: 497.821562ms FAIL (2025-07-08 05:11:41 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== GPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674108143 (355h49m20.600402838s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674596780 (355h49m20.600891471s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675604361 (355h49m20.601899058s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675768580 (355h49m20.602063277s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676120119 (355h49m20.60241481s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676215473 (355h49m20.602510169s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676623023 (355h49m20.602917716s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676647235 (355h49m20.602941929s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676696624 (355h49m20.602991321s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677177615 (355h49m20.603472312s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677707610 (355h49m20.604002306s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678250080 (355h49m20.604544775s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678907662 (355h49m20.605202358s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679453605 (355h49m20.6057483s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679618853 (355h49m20.605913551s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.771112310 (355h49m20.697407s) [upload=284997]: Next returned nil segment Jul 8 05:00:53.771144706 (355h49m20.697439396s) [upload=284997]: Waiting for error group managing segments... Jul 8 05:00:53.780217385 (355h49m20.70651207s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.780770161 (355h49m20.70706484s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781213801 (355h49m20.707508488s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781712476 (355h49m20.708007155s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.782192975 (355h49m20.708487647s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:01:23.612230509 (355h49m50.538525197s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612301272 (355h49m50.538595951s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 159.223.143.39:45850->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612349516 (355h49m50.538644191s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 159.223.143.39:38418->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612454902 (355h49m50.538749592s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612531645 (355h49m50.538826322s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 159.223.143.39:39432->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612552075 (355h49m50.538846753s) [noise=true , address=45.91.116.11:4118 , piece_id=KFPPGPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 159.223.143.39:33394->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612586657 (355h49m50.538881334s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 159.223.143.39:49070->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612604791 (355h49m50.538899468s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612642895 (355h49m50.538937572s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612651542 (355h49m50.538946218s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 159.223.143.39:47692->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612680902 (355h49m50.538975589s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612713168 (355h49m50.539007856s) [seg_pos={0 0}, upload=284997]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 8 05:01:23.648975097 (355h49m50.575269777s) [upload=284997]: Done uploading segments... Jul 8 05:01:23.649010451 (355h49m50.575305138s) [upload=284997]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 497.841236ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [...........................................XXX..............] duration: 470.941164ms FAIL (2025-07-08 05:11:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 8) ======LOGS====== pload=284991]: Uploading piece... Jul 8 05:00:37.137063656 (355h49m4.063358349s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.137984339 (355h49m4.064279032s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138034396 (355h49m4.064329092s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138836689 (355h49m4.065131384s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138884136 (355h49m4.065178829s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139428483 (355h49m4.065723178s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139436408 (355h49m4.0657311s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139788656 (355h49m4.066083348s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139920627 (355h49m4.066215325s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140071882 (355h49m4.066366572s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140353654 (355h49m4.06664834s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140415319 (355h49m4.066710015s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140681286 (355h49m4.066975982s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140973150 (355h49m4.067267847s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.144728804 (355h49m4.071023482s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.249883590 (355h49m4.176178265s) [upload=284991]: Next returned nil segment Jul 8 05:00:37.249902715 (355h49m4.176197412s) [upload=284991]: Waiting for error group managing segments... Jul 8 05:00:37.260212837 (355h49m4.186507526s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260803813 (355h49m4.187098502s) [noise=true , address=104.36.82.45:4915 , piece_id=G6IVO3DOQGN5GQKW67NX2ZKBOVTQEBA2PUNAGFDIKUI6OLPGMZHQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260834568 (355h49m4.187129264s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261367510 (355h49m4.187662188s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261455746 (355h49m4.187750422s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261903476 (355h49m4.188198155s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.278055422 (355h49m4.204350095s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.300910056 (355h49m4.227204736s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:01:37.042774308 (355h50m3.969069003s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:19LE2Lq4e6HojnMp9xQ3ySLkRe5PemXTwiQZThN5jT46bQhN6K, address:77.90.31.19:4855): protocol: rpc: dial tcp 77.90.31.19:4855: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042887419 (355h50m3.969182106s) [noise=true , address=77.90.31.16:4747 , piece_id=HV6IOGCMOHIUEYTGJMQ62V4UUNJ2MQD37SFF5Y23X2S66YYQSN3Q, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12ERdBm3XPw4s7JHeMWsVniwcqnfMtP4ZX8sBVjri6dzZtq8AF2, address:77.90.31.16:4747): protocol: rpc: dial tcp 77.90.31.16:4747: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042966233 (355h50m3.969260923s) [noise=true , address=77.90.31.19:4856 , piece_id=CR7AX6HZZGBQM5QATGLDLHQGUPZC4KW6H5CIBB4LFRTNL2WOLQFQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:1MntjJtcCDSRozKpesp6S2ZjYrMhxiQheDiPMPSCnavWBBGYt3, address:77.90.31.19:4856): protocol: rpc: dial tcp 77.90.31.19:4856: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043090813 (355h50m3.969385503s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: write tcp 159.223.143.39:47434->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043180860 (355h50m3.969475548s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12Ttdx6LHmuAt8JVC7arCDgWjm9eLadjeaEw17F12btoXuSiq9t, address:77.90.31.11:4583): protocol: write tcp 159.223.143.39:43298->77.90.31.11:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043193162 (355h50m3.969487876s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12LuFBTJg1aNmZoEdiQTtvfnfh5qnWVZFtqFX93L4fNfiYTYqJ6, address:77.90.31.18:4818): protocol: rpc: dial tcp 77.90.31.18:4818: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043283221 (355h50m3.969577895s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:126Sou5pnMm2eE2FJpv6WYX2VSqJvZZf9rFFJD6hFYDetRD28zu, address:77.90.31.12:4603): protocol: rpc: dial tcp 77.90.31.12:4603: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043307928 (355h50m3.969602654s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043494443 (355h50m3.969789132s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:168Tw7W69Tq9opDKhJ4cM8PGnzpxp9E9SxKbdVxJeF83pETqGx, address:77.90.31.17:4804): protocol: rpc: dial tcp 77.90.31.17:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043560259 (355h50m3.969854956s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: write tcp 159.223.143.39:37628->77.90.31.13:4632: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043598718 (355h50m3.969893411s) [seg_pos={0 0}, upload=284991]: Done waiting for segment. successful: 8 optimal: 14 errs: <nil> Jul 8 05:01:37.086050110 (355h50m4.012344788s) [upload=284991]: Done uploading segments... Jul 8 05:01:37.086091388 (355h50m4.012386065s) [upload=284991]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 8) Duration: 471.042077ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................................XXXX................] duration: 463.185242ms FAIL (2025-07-08 05:10:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== GPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674108143 (355h49m20.600402838s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674596780 (355h49m20.600891471s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675604361 (355h49m20.601899058s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675768580 (355h49m20.602063277s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676120119 (355h49m20.60241481s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676215473 (355h49m20.602510169s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676623023 (355h49m20.602917716s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676647235 (355h49m20.602941929s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676696624 (355h49m20.602991321s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677177615 (355h49m20.603472312s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677707610 (355h49m20.604002306s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678250080 (355h49m20.604544775s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678907662 (355h49m20.605202358s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679453605 (355h49m20.6057483s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679618853 (355h49m20.605913551s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.771112310 (355h49m20.697407s) [upload=284997]: Next returned nil segment Jul 8 05:00:53.771144706 (355h49m20.697439396s) [upload=284997]: Waiting for error group managing segments... Jul 8 05:00:53.780217385 (355h49m20.70651207s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.780770161 (355h49m20.70706484s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781213801 (355h49m20.707508488s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781712476 (355h49m20.708007155s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.782192975 (355h49m20.708487647s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:01:23.612230509 (355h49m50.538525197s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612301272 (355h49m50.538595951s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 159.223.143.39:45850->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612349516 (355h49m50.538644191s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 159.223.143.39:38418->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612454902 (355h49m50.538749592s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612531645 (355h49m50.538826322s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 159.223.143.39:39432->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612552075 (355h49m50.538846753s) [noise=true , address=45.91.116.11:4118 , piece_id=KFPPGPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 159.223.143.39:33394->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612586657 (355h49m50.538881334s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 159.223.143.39:49070->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612604791 (355h49m50.538899468s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612642895 (355h49m50.538937572s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612651542 (355h49m50.538946218s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 159.223.143.39:47692->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612680902 (355h49m50.538975589s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612713168 (355h49m50.539007856s) [seg_pos={0 0}, upload=284997]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 8 05:01:23.648975097 (355h49m50.575269777s) [upload=284997]: Done uploading segments... Jul 8 05:01:23.649010451 (355h49m50.575305138s) [upload=284997]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 463.302043ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [............................................XXX.............] duration: 470.146254ms FAIL (2025-07-08 05:10:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 8) ======LOGS====== pload=284991]: Uploading piece... Jul 8 05:00:37.137063656 (355h49m4.063358349s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.137984339 (355h49m4.064279032s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138034396 (355h49m4.064329092s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138836689 (355h49m4.065131384s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138884136 (355h49m4.065178829s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139428483 (355h49m4.065723178s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139436408 (355h49m4.0657311s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139788656 (355h49m4.066083348s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139920627 (355h49m4.066215325s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140071882 (355h49m4.066366572s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140353654 (355h49m4.06664834s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140415319 (355h49m4.066710015s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140681286 (355h49m4.066975982s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140973150 (355h49m4.067267847s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.144728804 (355h49m4.071023482s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.249883590 (355h49m4.176178265s) [upload=284991]: Next returned nil segment Jul 8 05:00:37.249902715 (355h49m4.176197412s) [upload=284991]: Waiting for error group managing segments... Jul 8 05:00:37.260212837 (355h49m4.186507526s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260803813 (355h49m4.187098502s) [noise=true , address=104.36.82.45:4915 , piece_id=G6IVO3DOQGN5GQKW67NX2ZKBOVTQEBA2PUNAGFDIKUI6OLPGMZHQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260834568 (355h49m4.187129264s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261367510 (355h49m4.187662188s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261455746 (355h49m4.187750422s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261903476 (355h49m4.188198155s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.278055422 (355h49m4.204350095s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.300910056 (355h49m4.227204736s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:01:37.042774308 (355h50m3.969069003s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:19LE2Lq4e6HojnMp9xQ3ySLkRe5PemXTwiQZThN5jT46bQhN6K, address:77.90.31.19:4855): protocol: rpc: dial tcp 77.90.31.19:4855: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042887419 (355h50m3.969182106s) [noise=true , address=77.90.31.16:4747 , piece_id=HV6IOGCMOHIUEYTGJMQ62V4UUNJ2MQD37SFF5Y23X2S66YYQSN3Q, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12ERdBm3XPw4s7JHeMWsVniwcqnfMtP4ZX8sBVjri6dzZtq8AF2, address:77.90.31.16:4747): protocol: rpc: dial tcp 77.90.31.16:4747: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042966233 (355h50m3.969260923s) [noise=true , address=77.90.31.19:4856 , piece_id=CR7AX6HZZGBQM5QATGLDLHQGUPZC4KW6H5CIBB4LFRTNL2WOLQFQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:1MntjJtcCDSRozKpesp6S2ZjYrMhxiQheDiPMPSCnavWBBGYt3, address:77.90.31.19:4856): protocol: rpc: dial tcp 77.90.31.19:4856: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043090813 (355h50m3.969385503s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: write tcp 159.223.143.39:47434->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043180860 (355h50m3.969475548s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12Ttdx6LHmuAt8JVC7arCDgWjm9eLadjeaEw17F12btoXuSiq9t, address:77.90.31.11:4583): protocol: write tcp 159.223.143.39:43298->77.90.31.11:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043193162 (355h50m3.969487876s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12LuFBTJg1aNmZoEdiQTtvfnfh5qnWVZFtqFX93L4fNfiYTYqJ6, address:77.90.31.18:4818): protocol: rpc: dial tcp 77.90.31.18:4818: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043283221 (355h50m3.969577895s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:126Sou5pnMm2eE2FJpv6WYX2VSqJvZZf9rFFJD6hFYDetRD28zu, address:77.90.31.12:4603): protocol: rpc: dial tcp 77.90.31.12:4603: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043307928 (355h50m3.969602654s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043494443 (355h50m3.969789132s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:168Tw7W69Tq9opDKhJ4cM8PGnzpxp9E9SxKbdVxJeF83pETqGx, address:77.90.31.17:4804): protocol: rpc: dial tcp 77.90.31.17:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043560259 (355h50m3.969854956s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: write tcp 159.223.143.39:37628->77.90.31.13:4632: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043598718 (355h50m3.969893411s) [seg_pos={0 0}, upload=284991]: Done waiting for segment. successful: 8 optimal: 14 errs: <nil> Jul 8 05:01:37.086050110 (355h50m4.012344788s) [upload=284991]: Done uploading segments... Jul 8 05:01:37.086091388 (355h50m4.012386065s) [upload=284991]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 8) Duration: 470.199731ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........................................XXXX...............] duration: 516.841167ms FAIL (2025-07-08 05:10:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== GPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674108143 (355h49m20.600402838s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674596780 (355h49m20.600891471s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675604361 (355h49m20.601899058s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675768580 (355h49m20.602063277s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676120119 (355h49m20.60241481s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676215473 (355h49m20.602510169s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676623023 (355h49m20.602917716s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676647235 (355h49m20.602941929s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676696624 (355h49m20.602991321s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677177615 (355h49m20.603472312s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677707610 (355h49m20.604002306s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678250080 (355h49m20.604544775s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678907662 (355h49m20.605202358s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679453605 (355h49m20.6057483s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679618853 (355h49m20.605913551s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.771112310 (355h49m20.697407s) [upload=284997]: Next returned nil segment Jul 8 05:00:53.771144706 (355h49m20.697439396s) [upload=284997]: Waiting for error group managing segments... Jul 8 05:00:53.780217385 (355h49m20.70651207s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.780770161 (355h49m20.70706484s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781213801 (355h49m20.707508488s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781712476 (355h49m20.708007155s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.782192975 (355h49m20.708487647s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:01:23.612230509 (355h49m50.538525197s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612301272 (355h49m50.538595951s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 159.223.143.39:45850->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612349516 (355h49m50.538644191s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 159.223.143.39:38418->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612454902 (355h49m50.538749592s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612531645 (355h49m50.538826322s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 159.223.143.39:39432->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612552075 (355h49m50.538846753s) [noise=true , address=45.91.116.11:4118 , piece_id=KFPPGPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 159.223.143.39:33394->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612586657 (355h49m50.538881334s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 159.223.143.39:49070->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612604791 (355h49m50.538899468s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612642895 (355h49m50.538937572s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612651542 (355h49m50.538946218s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 159.223.143.39:47692->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612680902 (355h49m50.538975589s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612713168 (355h49m50.539007856s) [seg_pos={0 0}, upload=284997]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 8 05:01:23.648975097 (355h49m50.575269777s) [upload=284997]: Done uploading segments... Jul 8 05:01:23.649010451 (355h49m50.575305138s) [upload=284997]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 516.87383ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.............................................XXX............] duration: 524.009392ms FAIL (2025-07-08 05:09:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 8) ======LOGS====== pload=284991]: Uploading piece... Jul 8 05:00:37.137063656 (355h49m4.063358349s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.137984339 (355h49m4.064279032s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138034396 (355h49m4.064329092s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138836689 (355h49m4.065131384s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138884136 (355h49m4.065178829s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139428483 (355h49m4.065723178s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139436408 (355h49m4.0657311s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139788656 (355h49m4.066083348s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139920627 (355h49m4.066215325s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140071882 (355h49m4.066366572s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140353654 (355h49m4.06664834s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140415319 (355h49m4.066710015s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140681286 (355h49m4.066975982s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140973150 (355h49m4.067267847s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.144728804 (355h49m4.071023482s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.249883590 (355h49m4.176178265s) [upload=284991]: Next returned nil segment Jul 8 05:00:37.249902715 (355h49m4.176197412s) [upload=284991]: Waiting for error group managing segments... Jul 8 05:00:37.260212837 (355h49m4.186507526s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260803813 (355h49m4.187098502s) [noise=true , address=104.36.82.45:4915 , piece_id=G6IVO3DOQGN5GQKW67NX2ZKBOVTQEBA2PUNAGFDIKUI6OLPGMZHQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260834568 (355h49m4.187129264s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261367510 (355h49m4.187662188s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261455746 (355h49m4.187750422s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261903476 (355h49m4.188198155s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.278055422 (355h49m4.204350095s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.300910056 (355h49m4.227204736s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:01:37.042774308 (355h50m3.969069003s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:19LE2Lq4e6HojnMp9xQ3ySLkRe5PemXTwiQZThN5jT46bQhN6K, address:77.90.31.19:4855): protocol: rpc: dial tcp 77.90.31.19:4855: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042887419 (355h50m3.969182106s) [noise=true , address=77.90.31.16:4747 , piece_id=HV6IOGCMOHIUEYTGJMQ62V4UUNJ2MQD37SFF5Y23X2S66YYQSN3Q, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12ERdBm3XPw4s7JHeMWsVniwcqnfMtP4ZX8sBVjri6dzZtq8AF2, address:77.90.31.16:4747): protocol: rpc: dial tcp 77.90.31.16:4747: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042966233 (355h50m3.969260923s) [noise=true , address=77.90.31.19:4856 , piece_id=CR7AX6HZZGBQM5QATGLDLHQGUPZC4KW6H5CIBB4LFRTNL2WOLQFQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:1MntjJtcCDSRozKpesp6S2ZjYrMhxiQheDiPMPSCnavWBBGYt3, address:77.90.31.19:4856): protocol: rpc: dial tcp 77.90.31.19:4856: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043090813 (355h50m3.969385503s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: write tcp 159.223.143.39:47434->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043180860 (355h50m3.969475548s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12Ttdx6LHmuAt8JVC7arCDgWjm9eLadjeaEw17F12btoXuSiq9t, address:77.90.31.11:4583): protocol: write tcp 159.223.143.39:43298->77.90.31.11:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043193162 (355h50m3.969487876s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12LuFBTJg1aNmZoEdiQTtvfnfh5qnWVZFtqFX93L4fNfiYTYqJ6, address:77.90.31.18:4818): protocol: rpc: dial tcp 77.90.31.18:4818: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043283221 (355h50m3.969577895s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:126Sou5pnMm2eE2FJpv6WYX2VSqJvZZf9rFFJD6hFYDetRD28zu, address:77.90.31.12:4603): protocol: rpc: dial tcp 77.90.31.12:4603: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043307928 (355h50m3.969602654s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043494443 (355h50m3.969789132s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:168Tw7W69Tq9opDKhJ4cM8PGnzpxp9E9SxKbdVxJeF83pETqGx, address:77.90.31.17:4804): protocol: rpc: dial tcp 77.90.31.17:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043560259 (355h50m3.969854956s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: write tcp 159.223.143.39:37628->77.90.31.13:4632: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043598718 (355h50m3.969893411s) [seg_pos={0 0}, upload=284991]: Done waiting for segment. successful: 8 optimal: 14 errs: <nil> Jul 8 05:01:37.086050110 (355h50m4.012344788s) [upload=284991]: Done uploading segments... Jul 8 05:01:37.086091388 (355h50m4.012386065s) [upload=284991]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 8) Duration: 524.036014ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........................................XXXX..............] duration: 450.5409ms FAIL (2025-07-08 05:09:38 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== GPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674108143 (355h49m20.600402838s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674596780 (355h49m20.600891471s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675604361 (355h49m20.601899058s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675768580 (355h49m20.602063277s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676120119 (355h49m20.60241481s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676215473 (355h49m20.602510169s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676623023 (355h49m20.602917716s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676647235 (355h49m20.602941929s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676696624 (355h49m20.602991321s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677177615 (355h49m20.603472312s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677707610 (355h49m20.604002306s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678250080 (355h49m20.604544775s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678907662 (355h49m20.605202358s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679453605 (355h49m20.6057483s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679618853 (355h49m20.605913551s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.771112310 (355h49m20.697407s) [upload=284997]: Next returned nil segment Jul 8 05:00:53.771144706 (355h49m20.697439396s) [upload=284997]: Waiting for error group managing segments... Jul 8 05:00:53.780217385 (355h49m20.70651207s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.780770161 (355h49m20.70706484s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781213801 (355h49m20.707508488s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781712476 (355h49m20.708007155s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.782192975 (355h49m20.708487647s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:01:23.612230509 (355h49m50.538525197s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612301272 (355h49m50.538595951s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 159.223.143.39:45850->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612349516 (355h49m50.538644191s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 159.223.143.39:38418->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612454902 (355h49m50.538749592s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612531645 (355h49m50.538826322s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 159.223.143.39:39432->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612552075 (355h49m50.538846753s) [noise=true , address=45.91.116.11:4118 , piece_id=KFPPGPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 159.223.143.39:33394->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612586657 (355h49m50.538881334s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 159.223.143.39:49070->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612604791 (355h49m50.538899468s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612642895 (355h49m50.538937572s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612651542 (355h49m50.538946218s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 159.223.143.39:47692->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612680902 (355h49m50.538975589s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612713168 (355h49m50.539007856s) [seg_pos={0 0}, upload=284997]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 8 05:01:23.648975097 (355h49m50.575269777s) [upload=284997]: Done uploading segments... Jul 8 05:01:23.649010451 (355h49m50.575305138s) [upload=284997]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 450.567339ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [..............................................XXX...........] duration: 432.815724ms FAIL (2025-07-08 05:09:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 8) ======LOGS====== pload=284991]: Uploading piece... Jul 8 05:00:37.137063656 (355h49m4.063358349s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.137984339 (355h49m4.064279032s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138034396 (355h49m4.064329092s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138836689 (355h49m4.065131384s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138884136 (355h49m4.065178829s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139428483 (355h49m4.065723178s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139436408 (355h49m4.0657311s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139788656 (355h49m4.066083348s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139920627 (355h49m4.066215325s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140071882 (355h49m4.066366572s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140353654 (355h49m4.06664834s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140415319 (355h49m4.066710015s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140681286 (355h49m4.066975982s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140973150 (355h49m4.067267847s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.144728804 (355h49m4.071023482s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.249883590 (355h49m4.176178265s) [upload=284991]: Next returned nil segment Jul 8 05:00:37.249902715 (355h49m4.176197412s) [upload=284991]: Waiting for error group managing segments... Jul 8 05:00:37.260212837 (355h49m4.186507526s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260803813 (355h49m4.187098502s) [noise=true , address=104.36.82.45:4915 , piece_id=G6IVO3DOQGN5GQKW67NX2ZKBOVTQEBA2PUNAGFDIKUI6OLPGMZHQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260834568 (355h49m4.187129264s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261367510 (355h49m4.187662188s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261455746 (355h49m4.187750422s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261903476 (355h49m4.188198155s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.278055422 (355h49m4.204350095s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.300910056 (355h49m4.227204736s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:01:37.042774308 (355h50m3.969069003s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:19LE2Lq4e6HojnMp9xQ3ySLkRe5PemXTwiQZThN5jT46bQhN6K, address:77.90.31.19:4855): protocol: rpc: dial tcp 77.90.31.19:4855: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042887419 (355h50m3.969182106s) [noise=true , address=77.90.31.16:4747 , piece_id=HV6IOGCMOHIUEYTGJMQ62V4UUNJ2MQD37SFF5Y23X2S66YYQSN3Q, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12ERdBm3XPw4s7JHeMWsVniwcqnfMtP4ZX8sBVjri6dzZtq8AF2, address:77.90.31.16:4747): protocol: rpc: dial tcp 77.90.31.16:4747: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042966233 (355h50m3.969260923s) [noise=true , address=77.90.31.19:4856 , piece_id=CR7AX6HZZGBQM5QATGLDLHQGUPZC4KW6H5CIBB4LFRTNL2WOLQFQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:1MntjJtcCDSRozKpesp6S2ZjYrMhxiQheDiPMPSCnavWBBGYt3, address:77.90.31.19:4856): protocol: rpc: dial tcp 77.90.31.19:4856: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043090813 (355h50m3.969385503s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: write tcp 159.223.143.39:47434->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043180860 (355h50m3.969475548s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12Ttdx6LHmuAt8JVC7arCDgWjm9eLadjeaEw17F12btoXuSiq9t, address:77.90.31.11:4583): protocol: write tcp 159.223.143.39:43298->77.90.31.11:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043193162 (355h50m3.969487876s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12LuFBTJg1aNmZoEdiQTtvfnfh5qnWVZFtqFX93L4fNfiYTYqJ6, address:77.90.31.18:4818): protocol: rpc: dial tcp 77.90.31.18:4818: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043283221 (355h50m3.969577895s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:126Sou5pnMm2eE2FJpv6WYX2VSqJvZZf9rFFJD6hFYDetRD28zu, address:77.90.31.12:4603): protocol: rpc: dial tcp 77.90.31.12:4603: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043307928 (355h50m3.969602654s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043494443 (355h50m3.969789132s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:168Tw7W69Tq9opDKhJ4cM8PGnzpxp9E9SxKbdVxJeF83pETqGx, address:77.90.31.17:4804): protocol: rpc: dial tcp 77.90.31.17:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043560259 (355h50m3.969854956s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: write tcp 159.223.143.39:37628->77.90.31.13:4632: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043598718 (355h50m3.969893411s) [seg_pos={0 0}, upload=284991]: Done waiting for segment. successful: 8 optimal: 14 errs: <nil> Jul 8 05:01:37.086050110 (355h50m4.012344788s) [upload=284991]: Done uploading segments... Jul 8 05:01:37.086091388 (355h50m4.012386065s) [upload=284991]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 8) Duration: 432.847ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........................................XXXX.............] duration: 535.482431ms FAIL (2025-07-08 05:09:02 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== GPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674108143 (355h49m20.600402838s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674596780 (355h49m20.600891471s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675604361 (355h49m20.601899058s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675768580 (355h49m20.602063277s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676120119 (355h49m20.60241481s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676215473 (355h49m20.602510169s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676623023 (355h49m20.602917716s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676647235 (355h49m20.602941929s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676696624 (355h49m20.602991321s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677177615 (355h49m20.603472312s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677707610 (355h49m20.604002306s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678250080 (355h49m20.604544775s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678907662 (355h49m20.605202358s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679453605 (355h49m20.6057483s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679618853 (355h49m20.605913551s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.771112310 (355h49m20.697407s) [upload=284997]: Next returned nil segment Jul 8 05:00:53.771144706 (355h49m20.697439396s) [upload=284997]: Waiting for error group managing segments... Jul 8 05:00:53.780217385 (355h49m20.70651207s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.780770161 (355h49m20.70706484s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781213801 (355h49m20.707508488s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781712476 (355h49m20.708007155s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.782192975 (355h49m20.708487647s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:01:23.612230509 (355h49m50.538525197s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612301272 (355h49m50.538595951s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 159.223.143.39:45850->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612349516 (355h49m50.538644191s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 159.223.143.39:38418->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612454902 (355h49m50.538749592s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612531645 (355h49m50.538826322s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 159.223.143.39:39432->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612552075 (355h49m50.538846753s) [noise=true , address=45.91.116.11:4118 , piece_id=KFPPGPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 159.223.143.39:33394->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612586657 (355h49m50.538881334s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 159.223.143.39:49070->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612604791 (355h49m50.538899468s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612642895 (355h49m50.538937572s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612651542 (355h49m50.538946218s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 159.223.143.39:47692->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612680902 (355h49m50.538975589s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612713168 (355h49m50.539007856s) [seg_pos={0 0}, upload=284997]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 8 05:01:23.648975097 (355h49m50.575269777s) [upload=284997]: Done uploading segments... Jul 8 05:01:23.649010451 (355h49m50.575305138s) [upload=284997]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 535.557511ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [............................................XXXX............] duration: 414.468797ms FAIL (2025-07-08 05:08:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== GPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674108143 (355h49m20.600402838s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674596780 (355h49m20.600891471s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675604361 (355h49m20.601899058s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675768580 (355h49m20.602063277s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676120119 (355h49m20.60241481s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676215473 (355h49m20.602510169s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676623023 (355h49m20.602917716s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676647235 (355h49m20.602941929s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676696624 (355h49m20.602991321s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677177615 (355h49m20.603472312s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677707610 (355h49m20.604002306s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678250080 (355h49m20.604544775s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678907662 (355h49m20.605202358s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679453605 (355h49m20.6057483s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679618853 (355h49m20.605913551s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.771112310 (355h49m20.697407s) [upload=284997]: Next returned nil segment Jul 8 05:00:53.771144706 (355h49m20.697439396s) [upload=284997]: Waiting for error group managing segments... Jul 8 05:00:53.780217385 (355h49m20.70651207s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.780770161 (355h49m20.70706484s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781213801 (355h49m20.707508488s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781712476 (355h49m20.708007155s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.782192975 (355h49m20.708487647s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:01:23.612230509 (355h49m50.538525197s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612301272 (355h49m50.538595951s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 159.223.143.39:45850->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612349516 (355h49m50.538644191s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 159.223.143.39:38418->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612454902 (355h49m50.538749592s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612531645 (355h49m50.538826322s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 159.223.143.39:39432->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612552075 (355h49m50.538846753s) [noise=true , address=45.91.116.11:4118 , piece_id=KFPPGPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 159.223.143.39:33394->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612586657 (355h49m50.538881334s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 159.223.143.39:49070->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612604791 (355h49m50.538899468s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612642895 (355h49m50.538937572s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612651542 (355h49m50.538946218s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 159.223.143.39:47692->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612680902 (355h49m50.538975589s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612713168 (355h49m50.539007856s) [seg_pos={0 0}, upload=284997]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 8 05:01:23.648975097 (355h49m50.575269777s) [upload=284997]: Done uploading segments... Jul 8 05:01:23.649010451 (355h49m50.575305138s) [upload=284997]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 414.492461ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [...............................................XXX..........] duration: 478.781672ms FAIL (2025-07-08 05:08:14 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 8) ======LOGS====== pload=284991]: Uploading piece... Jul 8 05:00:37.137063656 (355h49m4.063358349s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.137984339 (355h49m4.064279032s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138034396 (355h49m4.064329092s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138836689 (355h49m4.065131384s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138884136 (355h49m4.065178829s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139428483 (355h49m4.065723178s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139436408 (355h49m4.0657311s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139788656 (355h49m4.066083348s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139920627 (355h49m4.066215325s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140071882 (355h49m4.066366572s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140353654 (355h49m4.06664834s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140415319 (355h49m4.066710015s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140681286 (355h49m4.066975982s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140973150 (355h49m4.067267847s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.144728804 (355h49m4.071023482s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.249883590 (355h49m4.176178265s) [upload=284991]: Next returned nil segment Jul 8 05:00:37.249902715 (355h49m4.176197412s) [upload=284991]: Waiting for error group managing segments... Jul 8 05:00:37.260212837 (355h49m4.186507526s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260803813 (355h49m4.187098502s) [noise=true , address=104.36.82.45:4915 , piece_id=G6IVO3DOQGN5GQKW67NX2ZKBOVTQEBA2PUNAGFDIKUI6OLPGMZHQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260834568 (355h49m4.187129264s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261367510 (355h49m4.187662188s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261455746 (355h49m4.187750422s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261903476 (355h49m4.188198155s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.278055422 (355h49m4.204350095s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.300910056 (355h49m4.227204736s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:01:37.042774308 (355h50m3.969069003s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:19LE2Lq4e6HojnMp9xQ3ySLkRe5PemXTwiQZThN5jT46bQhN6K, address:77.90.31.19:4855): protocol: rpc: dial tcp 77.90.31.19:4855: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042887419 (355h50m3.969182106s) [noise=true , address=77.90.31.16:4747 , piece_id=HV6IOGCMOHIUEYTGJMQ62V4UUNJ2MQD37SFF5Y23X2S66YYQSN3Q, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12ERdBm3XPw4s7JHeMWsVniwcqnfMtP4ZX8sBVjri6dzZtq8AF2, address:77.90.31.16:4747): protocol: rpc: dial tcp 77.90.31.16:4747: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042966233 (355h50m3.969260923s) [noise=true , address=77.90.31.19:4856 , piece_id=CR7AX6HZZGBQM5QATGLDLHQGUPZC4KW6H5CIBB4LFRTNL2WOLQFQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:1MntjJtcCDSRozKpesp6S2ZjYrMhxiQheDiPMPSCnavWBBGYt3, address:77.90.31.19:4856): protocol: rpc: dial tcp 77.90.31.19:4856: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043090813 (355h50m3.969385503s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: write tcp 159.223.143.39:47434->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043180860 (355h50m3.969475548s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12Ttdx6LHmuAt8JVC7arCDgWjm9eLadjeaEw17F12btoXuSiq9t, address:77.90.31.11:4583): protocol: write tcp 159.223.143.39:43298->77.90.31.11:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043193162 (355h50m3.969487876s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12LuFBTJg1aNmZoEdiQTtvfnfh5qnWVZFtqFX93L4fNfiYTYqJ6, address:77.90.31.18:4818): protocol: rpc: dial tcp 77.90.31.18:4818: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043283221 (355h50m3.969577895s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:126Sou5pnMm2eE2FJpv6WYX2VSqJvZZf9rFFJD6hFYDetRD28zu, address:77.90.31.12:4603): protocol: rpc: dial tcp 77.90.31.12:4603: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043307928 (355h50m3.969602654s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043494443 (355h50m3.969789132s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:168Tw7W69Tq9opDKhJ4cM8PGnzpxp9E9SxKbdVxJeF83pETqGx, address:77.90.31.17:4804): protocol: rpc: dial tcp 77.90.31.17:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043560259 (355h50m3.969854956s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: write tcp 159.223.143.39:37628->77.90.31.13:4632: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043598718 (355h50m3.969893411s) [seg_pos={0 0}, upload=284991]: Done waiting for segment. successful: 8 optimal: 14 errs: <nil> Jul 8 05:01:37.086050110 (355h50m4.012344788s) [upload=284991]: Done uploading segments... Jul 8 05:01:37.086091388 (355h50m4.012386065s) [upload=284991]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 8) Duration: 478.816769ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............................................XXXX...........] duration: 455.21217ms FAIL (2025-07-08 05:08:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== GPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674108143 (355h49m20.600402838s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674596780 (355h49m20.600891471s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675604361 (355h49m20.601899058s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675768580 (355h49m20.602063277s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676120119 (355h49m20.60241481s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676215473 (355h49m20.602510169s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676623023 (355h49m20.602917716s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676647235 (355h49m20.602941929s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676696624 (355h49m20.602991321s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677177615 (355h49m20.603472312s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677707610 (355h49m20.604002306s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678250080 (355h49m20.604544775s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678907662 (355h49m20.605202358s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679453605 (355h49m20.6057483s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679618853 (355h49m20.605913551s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.771112310 (355h49m20.697407s) [upload=284997]: Next returned nil segment Jul 8 05:00:53.771144706 (355h49m20.697439396s) [upload=284997]: Waiting for error group managing segments... Jul 8 05:00:53.780217385 (355h49m20.70651207s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.780770161 (355h49m20.70706484s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781213801 (355h49m20.707508488s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781712476 (355h49m20.708007155s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.782192975 (355h49m20.708487647s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:01:23.612230509 (355h49m50.538525197s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612301272 (355h49m50.538595951s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 159.223.143.39:45850->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612349516 (355h49m50.538644191s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 159.223.143.39:38418->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612454902 (355h49m50.538749592s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612531645 (355h49m50.538826322s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 159.223.143.39:39432->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612552075 (355h49m50.538846753s) [noise=true , address=45.91.116.11:4118 , piece_id=KFPPGPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 159.223.143.39:33394->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612586657 (355h49m50.538881334s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 159.223.143.39:49070->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612604791 (355h49m50.538899468s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612642895 (355h49m50.538937572s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612651542 (355h49m50.538946218s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 159.223.143.39:47692->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612680902 (355h49m50.538975589s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612713168 (355h49m50.539007856s) [seg_pos={0 0}, upload=284997]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 8 05:01:23.648975097 (355h49m50.575269777s) [upload=284997]: Done uploading segments... Jul 8 05:01:23.649010451 (355h49m50.575305138s) [upload=284997]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 455.240995ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [................................................XXX.........] duration: 635.970562ms FAIL (2025-07-08 05:07:43 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 8) ======LOGS====== pload=284991]: Uploading piece... Jul 8 05:00:37.137063656 (355h49m4.063358349s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.137984339 (355h49m4.064279032s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138034396 (355h49m4.064329092s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138836689 (355h49m4.065131384s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138884136 (355h49m4.065178829s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139428483 (355h49m4.065723178s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139436408 (355h49m4.0657311s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139788656 (355h49m4.066083348s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139920627 (355h49m4.066215325s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140071882 (355h49m4.066366572s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140353654 (355h49m4.06664834s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140415319 (355h49m4.066710015s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140681286 (355h49m4.066975982s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140973150 (355h49m4.067267847s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.144728804 (355h49m4.071023482s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.249883590 (355h49m4.176178265s) [upload=284991]: Next returned nil segment Jul 8 05:00:37.249902715 (355h49m4.176197412s) [upload=284991]: Waiting for error group managing segments... Jul 8 05:00:37.260212837 (355h49m4.186507526s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260803813 (355h49m4.187098502s) [noise=true , address=104.36.82.45:4915 , piece_id=G6IVO3DOQGN5GQKW67NX2ZKBOVTQEBA2PUNAGFDIKUI6OLPGMZHQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260834568 (355h49m4.187129264s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261367510 (355h49m4.187662188s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261455746 (355h49m4.187750422s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261903476 (355h49m4.188198155s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.278055422 (355h49m4.204350095s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.300910056 (355h49m4.227204736s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:01:37.042774308 (355h50m3.969069003s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:19LE2Lq4e6HojnMp9xQ3ySLkRe5PemXTwiQZThN5jT46bQhN6K, address:77.90.31.19:4855): protocol: rpc: dial tcp 77.90.31.19:4855: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042887419 (355h50m3.969182106s) [noise=true , address=77.90.31.16:4747 , piece_id=HV6IOGCMOHIUEYTGJMQ62V4UUNJ2MQD37SFF5Y23X2S66YYQSN3Q, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12ERdBm3XPw4s7JHeMWsVniwcqnfMtP4ZX8sBVjri6dzZtq8AF2, address:77.90.31.16:4747): protocol: rpc: dial tcp 77.90.31.16:4747: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042966233 (355h50m3.969260923s) [noise=true , address=77.90.31.19:4856 , piece_id=CR7AX6HZZGBQM5QATGLDLHQGUPZC4KW6H5CIBB4LFRTNL2WOLQFQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:1MntjJtcCDSRozKpesp6S2ZjYrMhxiQheDiPMPSCnavWBBGYt3, address:77.90.31.19:4856): protocol: rpc: dial tcp 77.90.31.19:4856: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043090813 (355h50m3.969385503s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: write tcp 159.223.143.39:47434->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043180860 (355h50m3.969475548s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12Ttdx6LHmuAt8JVC7arCDgWjm9eLadjeaEw17F12btoXuSiq9t, address:77.90.31.11:4583): protocol: write tcp 159.223.143.39:43298->77.90.31.11:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043193162 (355h50m3.969487876s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12LuFBTJg1aNmZoEdiQTtvfnfh5qnWVZFtqFX93L4fNfiYTYqJ6, address:77.90.31.18:4818): protocol: rpc: dial tcp 77.90.31.18:4818: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043283221 (355h50m3.969577895s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:126Sou5pnMm2eE2FJpv6WYX2VSqJvZZf9rFFJD6hFYDetRD28zu, address:77.90.31.12:4603): protocol: rpc: dial tcp 77.90.31.12:4603: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043307928 (355h50m3.969602654s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043494443 (355h50m3.969789132s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:168Tw7W69Tq9opDKhJ4cM8PGnzpxp9E9SxKbdVxJeF83pETqGx, address:77.90.31.17:4804): protocol: rpc: dial tcp 77.90.31.17:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043560259 (355h50m3.969854956s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: write tcp 159.223.143.39:37628->77.90.31.13:4632: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043598718 (355h50m3.969893411s) [seg_pos={0 0}, upload=284991]: Done waiting for segment. successful: 8 optimal: 14 errs: <nil> Jul 8 05:01:37.086050110 (355h50m4.012344788s) [upload=284991]: Done uploading segments... Jul 8 05:01:37.086091388 (355h50m4.012386065s) [upload=284991]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 8) Duration: 636.05538ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............................................XXXX..........] duration: 416.385257ms FAIL (2025-07-08 05:07:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== GPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674108143 (355h49m20.600402838s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674596780 (355h49m20.600891471s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675604361 (355h49m20.601899058s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675768580 (355h49m20.602063277s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676120119 (355h49m20.60241481s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676215473 (355h49m20.602510169s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676623023 (355h49m20.602917716s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676647235 (355h49m20.602941929s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676696624 (355h49m20.602991321s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677177615 (355h49m20.603472312s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677707610 (355h49m20.604002306s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678250080 (355h49m20.604544775s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678907662 (355h49m20.605202358s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679453605 (355h49m20.6057483s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679618853 (355h49m20.605913551s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.771112310 (355h49m20.697407s) [upload=284997]: Next returned nil segment Jul 8 05:00:53.771144706 (355h49m20.697439396s) [upload=284997]: Waiting for error group managing segments... Jul 8 05:00:53.780217385 (355h49m20.70651207s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.780770161 (355h49m20.70706484s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781213801 (355h49m20.707508488s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781712476 (355h49m20.708007155s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.782192975 (355h49m20.708487647s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:01:23.612230509 (355h49m50.538525197s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612301272 (355h49m50.538595951s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 159.223.143.39:45850->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612349516 (355h49m50.538644191s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 159.223.143.39:38418->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612454902 (355h49m50.538749592s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612531645 (355h49m50.538826322s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 159.223.143.39:39432->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612552075 (355h49m50.538846753s) [noise=true , address=45.91.116.11:4118 , piece_id=KFPPGPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 159.223.143.39:33394->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612586657 (355h49m50.538881334s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 159.223.143.39:49070->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612604791 (355h49m50.538899468s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612642895 (355h49m50.538937572s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612651542 (355h49m50.538946218s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 159.223.143.39:47692->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612680902 (355h49m50.538975589s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612713168 (355h49m50.539007856s) [seg_pos={0 0}, upload=284997]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 8 05:01:23.648975097 (355h49m50.575269777s) [upload=284997]: Done uploading segments... Jul 8 05:01:23.649010451 (355h49m50.575305138s) [upload=284997]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 416.41161ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.................................................XXX........] duration: 453.254266ms FAIL (2025-07-08 05:07:13 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 8) ======LOGS====== pload=284991]: Uploading piece... Jul 8 05:00:37.137063656 (355h49m4.063358349s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.137984339 (355h49m4.064279032s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138034396 (355h49m4.064329092s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138836689 (355h49m4.065131384s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.138884136 (355h49m4.065178829s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139428483 (355h49m4.065723178s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139436408 (355h49m4.0657311s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139788656 (355h49m4.066083348s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.139920627 (355h49m4.066215325s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140071882 (355h49m4.066366572s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140353654 (355h49m4.06664834s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140415319 (355h49m4.066710015s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140681286 (355h49m4.066975982s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.140973150 (355h49m4.067267847s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.144728804 (355h49m4.071023482s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Uploading piece... Jul 8 05:00:37.249883590 (355h49m4.176178265s) [upload=284991]: Next returned nil segment Jul 8 05:00:37.249902715 (355h49m4.176197412s) [upload=284991]: Waiting for error group managing segments... Jul 8 05:00:37.260212837 (355h49m4.186507526s) [noise=true , address=104.36.82.21:4505 , piece_id=COWHHW42Y6SQ5MPFTYWBMFLRVG5IPODENUDMHVIP4C6ZCQ255AWA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260803813 (355h49m4.187098502s) [noise=true , address=104.36.82.45:4915 , piece_id=G6IVO3DOQGN5GQKW67NX2ZKBOVTQEBA2PUNAGFDIKUI6OLPGMZHQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.260834568 (355h49m4.187129264s) [noise=true , address=104.36.82.37:4680 , piece_id=OJJ2PZASYAZOYLIKMPNIPU4X26NNOHVZTO3OGH2J2XASO2IVAIGA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261367510 (355h49m4.187662188s) [noise=true , address=104.36.82.19:4436 , piece_id=S2JCPQDCWXZGPDROJEKK5KL4ZS2HWY5GTPKH4EJMK5DJ6BDWGPNA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261455746 (355h49m4.187750422s) [noise=true , address=104.36.82.39:4754 , piece_id=KMKEARZDE7WLZAFNPB6MHKKVMAMSN4JCZWVU447JSTK63KODGTKQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.261903476 (355h49m4.188198155s) [noise=true , address=104.36.82.44:4881 , piece_id=GJQAUZWBAMKO3DJTFMES2AU5FL5J74VUGJ5O7DBMN3MFVDHA6FQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.278055422 (355h49m4.204350095s) [noise=true , address=104.36.82.60:4360 , piece_id=DBJIWOXACPT3GK66GZYTDT5PGN3WEGOD7UZUQW2K7JGTMYB3M23A, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:00:37.300910056 (355h49m4.227204736s) [noise=true , address=104.36.82.23:4464 , piece_id=JHDO2RBBZ2RIDLZ743E6X2MWOW3PQFEPOX2SSHEW3JDXPSXBYUDA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: <nil> Jul 8 05:01:37.042774308 (355h50m3.969069003s) [noise=true , address=77.90.31.19:4855 , piece_id=46RSC2UCI5QVLZJEHAU5S245IGML6AMJRDATHCTPWF4242SRI5RQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:19LE2Lq4e6HojnMp9xQ3ySLkRe5PemXTwiQZThN5jT46bQhN6K, address:77.90.31.19:4855): protocol: rpc: dial tcp 77.90.31.19:4855: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042887419 (355h50m3.969182106s) [noise=true , address=77.90.31.16:4747 , piece_id=HV6IOGCMOHIUEYTGJMQ62V4UUNJ2MQD37SFF5Y23X2S66YYQSN3Q, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12ERdBm3XPw4s7JHeMWsVniwcqnfMtP4ZX8sBVjri6dzZtq8AF2, address:77.90.31.16:4747): protocol: rpc: dial tcp 77.90.31.16:4747: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.042966233 (355h50m3.969260923s) [noise=true , address=77.90.31.19:4856 , piece_id=CR7AX6HZZGBQM5QATGLDLHQGUPZC4KW6H5CIBB4LFRTNL2WOLQFQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:1MntjJtcCDSRozKpesp6S2ZjYrMhxiQheDiPMPSCnavWBBGYt3, address:77.90.31.19:4856): protocol: rpc: dial tcp 77.90.31.19:4856: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043090813 (355h50m3.969385503s) [noise=true , address=77.90.31.14:4664 , piece_id=ESB7MNGQ7H6RD4P2IXUDWEZJARODYVQW3TC4NI64NMMGEEPBBFPA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: write tcp 159.223.143.39:47434->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043180860 (355h50m3.969475548s) [noise=true , address=77.90.31.11:4583 , piece_id=WX6UA42RKXN7SPI2RB4NXXY73PFK6LRLXVWIFAOKHLBYYITHDUCA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12Ttdx6LHmuAt8JVC7arCDgWjm9eLadjeaEw17F12btoXuSiq9t, address:77.90.31.11:4583): protocol: write tcp 159.223.143.39:43298->77.90.31.11:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043193162 (355h50m3.969487876s) [noise=true , address=77.90.31.18:4818 , piece_id=2GIDSTCP3A442OS44ZIG5KUMB7Y5B4337WAS32XLE5ZRNGU4Z4PQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12LuFBTJg1aNmZoEdiQTtvfnfh5qnWVZFtqFX93L4fNfiYTYqJ6, address:77.90.31.18:4818): protocol: rpc: dial tcp 77.90.31.18:4818: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043283221 (355h50m3.969577895s) [noise=true , address=77.90.31.12:4603 , piece_id=7MCDNMIXJVDLTTWA3TBEDYKDLSXCN36JNHK7SSBDFDGU6XDWXXBA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:126Sou5pnMm2eE2FJpv6WYX2VSqJvZZf9rFFJD6hFYDetRD28zu, address:77.90.31.12:4603): protocol: rpc: dial tcp 77.90.31.12:4603: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043307928 (355h50m3.969602654s) [noise=true , address=77.90.31.21:4895 , piece_id=RY6MY6WBPUAGBQYGW6NS4NKGKTNAF5NUGOBNAVGFNLJHYGCWPZMQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: rpc: dial tcp 77.90.31.21:4895: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043494443 (355h50m3.969789132s) [noise=true , address=77.90.31.17:4804 , piece_id=IUPYWS2X4ECV7LZ2M2WDMKONPVVD6T5SHUEX3747I4X7B54FPNRQ, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:168Tw7W69Tq9opDKhJ4cM8PGnzpxp9E9SxKbdVxJeF83pETqGx, address:77.90.31.17:4804): protocol: rpc: dial tcp 77.90.31.17:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043560259 (355h50m3.969854956s) [noise=true , address=77.90.31.13:4632 , piece_id=5U23ECS42TH66KHWTI27WZE5ASUHGNRB2NFEVS6IOW44DY76DPQA, seg_pos={0 0}, upload=284991]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: write tcp 159.223.143.39:37628->77.90.31.13:4632: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:37.043598718 (355h50m3.969893411s) [seg_pos={0 0}, upload=284991]: Done waiting for segment. successful: 8 optimal: 14 errs: <nil> Jul 8 05:01:37.086050110 (355h50m4.012344788s) [upload=284991]: Done uploading segments... Jul 8 05:01:37.086091388 (355h50m4.012386065s) [upload=284991]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 8) Duration: 453.290617ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............................................XXXX.........] duration: 490.926591ms FAIL (2025-07-08 05:07:12 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== GPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674108143 (355h49m20.600402838s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674596780 (355h49m20.600891471s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675604361 (355h49m20.601899058s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675768580 (355h49m20.602063277s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676120119 (355h49m20.60241481s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676215473 (355h49m20.602510169s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676623023 (355h49m20.602917716s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676647235 (355h49m20.602941929s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676696624 (355h49m20.602991321s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677177615 (355h49m20.603472312s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677707610 (355h49m20.604002306s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678250080 (355h49m20.604544775s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678907662 (355h49m20.605202358s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679453605 (355h49m20.6057483s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679618853 (355h49m20.605913551s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.771112310 (355h49m20.697407s) [upload=284997]: Next returned nil segment Jul 8 05:00:53.771144706 (355h49m20.697439396s) [upload=284997]: Waiting for error group managing segments... Jul 8 05:00:53.780217385 (355h49m20.70651207s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.780770161 (355h49m20.70706484s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781213801 (355h49m20.707508488s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781712476 (355h49m20.708007155s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.782192975 (355h49m20.708487647s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:01:23.612230509 (355h49m50.538525197s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612301272 (355h49m50.538595951s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 159.223.143.39:45850->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612349516 (355h49m50.538644191s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 159.223.143.39:38418->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612454902 (355h49m50.538749592s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612531645 (355h49m50.538826322s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 159.223.143.39:39432->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612552075 (355h49m50.538846753s) [noise=true , address=45.91.116.11:4118 , piece_id=KFPPGPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 159.223.143.39:33394->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612586657 (355h49m50.538881334s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 159.223.143.39:49070->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612604791 (355h49m50.538899468s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612642895 (355h49m50.538937572s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612651542 (355h49m50.538946218s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 159.223.143.39:47692->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612680902 (355h49m50.538975589s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612713168 (355h49m50.539007856s) [seg_pos={0 0}, upload=284997]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 8 05:01:23.648975097 (355h49m50.575269777s) [upload=284997]: Done uploading segments... Jul 8 05:01:23.649010451 (355h49m50.575305138s) [upload=284997]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 490.950878ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [................................................XXXX........] duration: 446.348178ms FAIL (2025-07-08 05:06:51 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== GPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674108143 (355h49m20.600402838s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.674596780 (355h49m20.600891471s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675604361 (355h49m20.601899058s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.675768580 (355h49m20.602063277s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676120119 (355h49m20.60241481s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676215473 (355h49m20.602510169s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676623023 (355h49m20.602917716s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676647235 (355h49m20.602941929s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.676696624 (355h49m20.602991321s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677177615 (355h49m20.603472312s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.677707610 (355h49m20.604002306s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678250080 (355h49m20.604544775s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.678907662 (355h49m20.605202358s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679453605 (355h49m20.6057483s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.679618853 (355h49m20.605913551s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Uploading piece... Jul 8 05:00:53.771112310 (355h49m20.697407s) [upload=284997]: Next returned nil segment Jul 8 05:00:53.771144706 (355h49m20.697439396s) [upload=284997]: Waiting for error group managing segments... Jul 8 05:00:53.780217385 (355h49m20.70651207s) [noise=true , address=104.36.82.16:4294 , piece_id=UMOOA2KAWCCQ2ADFRU5KMF5C2QKQSSG2BUBWFHHFYLN34D6ZTZNQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.780770161 (355h49m20.70706484s) [noise=true , address=104.36.82.14:4223 , piece_id=MN6ZYN6WRG24ZWUCWKHWKWJIIFIOTVCMD2LZRVZYN7YFJF2HHWVQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781213801 (355h49m20.707508488s) [noise=true , address=104.36.82.7:4909 , piece_id=IVYHKYHXY6KIQZVLEAFOBUQDPMFQKWGXE5G7TMAP4DIOF2RWSFTQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.781712476 (355h49m20.708007155s) [noise=true , address=104.36.82.8:4972 , piece_id=SYE6RITGGKDRP2L3B5CG7TSOWAKW2IUHOOZMDCIDMW7BMRHN6T2A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:00:53.782192975 (355h49m20.708487647s) [noise=true , address=104.36.82.9:4005 , piece_id=OYXXEPKSZY76SYXGFJULBCHYBBUYBRZQNE35CHJWBSUPP3HYEKXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: <nil> Jul 8 05:01:23.612230509 (355h49m50.538525197s) [noise=true , address=45.91.116.7:4010 , piece_id=MVNG6RHZXKDER3V25KSLORVSTUP2XOONSXLM6XHI33S4AYYLQAZA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612301272 (355h49m50.538595951s) [noise=true , address=77.90.31.12:4589 , piece_id=LEZMVCMGJUZZ6EKTFLUU3HLBPWISRWRQROE755DMJSLAOFJMO7SA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 159.223.143.39:45850->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612349516 (355h49m50.538644191s) [noise=true , address=45.91.116.4:4938 , piece_id=GTFP7GQZF7DSS4RGA5YMBAYH2RP2BZT2PZG7CMGD4C7IOJQNIICQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 159.223.143.39:38418->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612454902 (355h49m50.538749592s) [noise=true , address=77.90.31.14:4662 , piece_id=OFALEXDRGT6DMUZ4T73HQTYKII7CJFYSBFYGCVEJDODZ5XXJUPXA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612531645 (355h49m50.538826322s) [noise=true , address=45.91.116.12:4227 , piece_id=QEAVJTXOWMVH7JCF3BAPA7IKYGRDMCQUEPZM7UNQ5TLH577UUTXQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: write tcp 159.223.143.39:39432->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612552075 (355h49m50.538846753s) [noise=true , address=45.91.116.11:4118 , piece_id=KFPPGPWOS7PDYBGT4QXOFZOJ5GDXDAAGR2VONXWZ4EDZXN4IZJDQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 159.223.143.39:33394->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612586657 (355h49m50.538881334s) [noise=true , address=45.91.116.9:4262 , piece_id=GMZSOJJJGC7Z5CKIXLQFKTKPSUT6ONGFKNG6LAU52O55RBL7GWKA, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 159.223.143.39:49070->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612604791 (355h49m50.538899468s) [noise=true , address=45.91.116.13:4153 , piece_id=IMG3DME7CFJM75LESTOLFWXCU3WNJUMNUUFRUKHFDREDYIN32XEQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612642895 (355h49m50.538937572s) [noise=true , address=45.91.116.14:4191 , piece_id=KQLOAIEJBPSCFGFKBV64B5UKC5TUDF22WXFKFPIC3KB7YHLBVY4A, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612651542 (355h49m50.538946218s) [noise=true , address=77.90.31.17:4772 , piece_id=UIYNDDDBEDWKRRQNJF2IIDH2WUBW2AZOV2356NWXW7JIOX3B3SHQ, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 159.223.143.39:47692->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612680902 (355h49m50.538975589s) [noise=true , address=45.91.116.5:4299 , piece_id=Z2EMWRSBVV3NUQAAEJ3RKKSZA64LWM75YJQT24WJF4HCZF4XHG6Q, seg_pos={0 0}, upload=284997]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 8 05:01:23.612713168 (355h49m50.539007856s) [seg_pos={0 0}, upload=284997]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 8 05:01:23.648975097 (355h49m50.575269777s) [upload=284997]: Done uploading segments... Jul 8 05:01:23.649010451 (355h49m50.575305138s) [upload=284997]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 446.381016ms core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)