Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 557.035531ms FAIL (2025-06-21 12:41:18 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 557.061698ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 464.879348ms FAIL (2025-06-21 12:40:43 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 464.91109ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 469.694811ms FAIL (2025-06-21 12:40:26 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 469.723866ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 436.908602ms FAIL (2025-06-21 12:39:20 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 436.938524ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 445.228059ms FAIL (2025-06-21 12:38:36 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 445.281734ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 557.045759ms FAIL (2025-06-21 12:37:24 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 557.076322ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 421.25698ms FAIL (2025-06-21 12:36:21 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 421.283531ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 477.90386ms FAIL (2025-06-21 12:35:54 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 477.95157ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 479.21863ms FAIL (2025-06-21 12:34:46 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 479.248074ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 522.408568ms FAIL (2025-06-21 12:33:36 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 522.434605ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 375.533558ms FAIL (2025-06-21 12:32:28 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 375.5559ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 539.432113ms FAIL (2025-06-21 12:31:50 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 539.464129ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 494.031865ms FAIL (2025-06-21 12:30:41 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 494.076682ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 509.927431ms FAIL (2025-06-21 12:30:01 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 509.953883ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 562.97905ms FAIL (2025-06-21 12:29:38 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 563.006028ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 480.118102ms FAIL (2025-06-21 12:28:26 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 480.225245ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 523.951641ms FAIL (2025-06-21 12:28:11 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 523.98429ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 713.907731ms FAIL (2025-06-21 12:27:08 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 713.939157ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 560.02687ms FAIL (2025-06-21 12:26:34 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 560.056065ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 598.008622ms FAIL (2025-06-21 12:26:10 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 598.038204ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 465.338673ms FAIL (2025-06-21 12:25:21 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 465.372042ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 581.321547ms FAIL (2025-06-21 12:25:03 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 581.357814ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 493.715689ms FAIL (2025-06-21 12:24:38 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 493.753332ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 499.910661ms FAIL (2025-06-21 12:24:04 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 499.943077ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 518.185293ms FAIL (2025-06-21 12:23:16 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 518.220604ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 510.959451ms FAIL (2025-06-21 12:22:53 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 510.999681ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 410.463097ms FAIL (2025-06-21 12:22:10 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 410.497408ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 523.255412ms FAIL (2025-06-21 12:21:31 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 523.284438ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 463.317156ms FAIL (2025-06-21 12:21:11 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 463.366975ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 729.225937ms FAIL (2025-06-21 12:20:54 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 729.257092ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 518.507519ms FAIL (2025-06-21 12:19:47 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 518.531515ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 417.218583ms FAIL (2025-06-21 12:19:27 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 417.253013ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 466.678915ms FAIL (2025-06-21 12:18:57 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 466.711314ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 533.911344ms FAIL (2025-06-21 12:18:18 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 533.943322ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 580.0224ms FAIL (2025-06-21 12:17:45 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 580.053079ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 477.067544ms FAIL (2025-06-21 12:16:34 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 477.163139ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 481.391214ms FAIL (2025-06-21 12:15:48 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 481.422983ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 472.606518ms FAIL (2025-06-21 12:14:48 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 472.638774ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 483.488834ms FAIL (2025-06-21 12:14:01 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 483.524406ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 514.220323ms FAIL (2025-06-21 12:13:41 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 514.250539ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 458.259622ms FAIL (2025-06-21 12:13:02 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 458.31076ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 443.155287ms FAIL (2025-06-21 12:12:35 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 443.197212ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 604.798173ms FAIL (2025-06-21 12:11:32 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 604.832338ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 557.497321ms FAIL (2025-06-21 12:10:25 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 557.524911ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 404.015565ms FAIL (2025-06-21 12:09:54 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 404.046722ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 481.727122ms FAIL (2025-06-21 12:08:48 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 481.753177ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 589.356783ms FAIL (2025-06-21 12:08:16 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 589.404392ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 473.325792ms FAIL (2025-06-21 12:07:42 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 473.359931ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 461.580558ms FAIL (2025-06-21 12:06:29 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 461.61356ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 501.796887ms FAIL (2025-06-21 12:06:12 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 501.826087ms core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)