Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 499.910661ms FAIL (2025-06-21 12:24:04 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 499.943077ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 518.185293ms FAIL (2025-06-21 12:23:16 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 518.220604ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 510.959451ms FAIL (2025-06-21 12:22:53 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 510.999681ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 410.463097ms FAIL (2025-06-21 12:22:10 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 410.497408ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 523.255412ms FAIL (2025-06-21 12:21:31 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 523.284438ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 463.317156ms FAIL (2025-06-21 12:21:11 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 463.366975ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 729.225937ms FAIL (2025-06-21 12:20:54 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 729.257092ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 518.507519ms FAIL (2025-06-21 12:19:47 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 518.531515ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 417.218583ms FAIL (2025-06-21 12:19:27 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 417.253013ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 466.678915ms FAIL (2025-06-21 12:18:57 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 466.711314ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 533.911344ms FAIL (2025-06-21 12:18:18 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 533.943322ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 580.0224ms FAIL (2025-06-21 12:17:45 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 580.053079ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 477.067544ms FAIL (2025-06-21 12:16:34 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 477.163139ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 481.391214ms FAIL (2025-06-21 12:15:48 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 481.422983ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 472.606518ms FAIL (2025-06-21 12:14:48 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 472.638774ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 483.488834ms FAIL (2025-06-21 12:14:01 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 483.524406ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 514.220323ms FAIL (2025-06-21 12:13:41 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 514.250539ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 458.259622ms FAIL (2025-06-21 12:13:02 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 458.31076ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 443.155287ms FAIL (2025-06-21 12:12:35 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 443.197212ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 604.798173ms FAIL (2025-06-21 12:11:32 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 604.832338ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 557.497321ms FAIL (2025-06-21 12:10:25 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 557.524911ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 404.015565ms FAIL (2025-06-21 12:09:54 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 404.046722ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 481.727122ms FAIL (2025-06-21 12:08:48 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 481.753177ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 589.356783ms FAIL (2025-06-21 12:08:16 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 589.404392ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 473.325792ms FAIL (2025-06-21 12:07:42 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 473.359931ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 461.580558ms FAIL (2025-06-21 12:06:29 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 461.61356ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 501.796887ms FAIL (2025-06-21 12:06:12 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 501.826087ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 470.397198ms FAIL (2025-06-21 12:05:48 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 470.522089ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 586.822044ms FAIL (2025-06-21 12:04:58 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 586.862528ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 556.573749ms FAIL (2025-06-21 12:03:57 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 556.613146ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 617.67072ms FAIL (2025-06-21 12:03:17 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 617.711492ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 397.575876ms FAIL (2025-06-21 12:02:55 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 7h38m7.375709734s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 21 12:01:31.854094816 (67h38m7.417417807s) [seg_pos={0 0}, upload=54163]: Begin upload segment... Jun 21 12:01:31.854161536 (67h38m7.417484526s) [seg_pos={0 0}, upload=54163]: Done begin upload segment. Jun 21 12:01:31.854200300 (67h38m7.417523301s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854207617 (67h38m7.417530629s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854807169 (67h38m7.418130179s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.854829748 (67h38m7.418152758s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855329210 (67h38m7.418652221s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855490275 (67h38m7.418813287s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855724778 (67h38m7.419047789s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.855890677 (67h38m7.419213687s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856338104 (67h38m7.419661107s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856688746 (67h38m7.420011743s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.856750875 (67h38m7.42007388s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857212611 (67h38m7.420535623s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.857664744 (67h38m7.420987747s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.858206960 (67h38m7.421529972s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.859952413 (67h38m7.423275423s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860426771 (67h38m7.423749785s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.860698201 (67h38m7.424021214s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.861239830 (67h38m7.424562829s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Uploading piece... Jun 21 12:01:31.963992975 (67h38m7.527315966s) [upload=54163]: Next returned nil segment Jun 21 12:01:31.968834670 (67h38m7.532157682s) [upload=54163]: Waiting for error group managing segments... Jun 21 12:01:31.970421216 (67h38m7.533744219s) [noise=true , address=79.127.219.48:4082 , piece_id=7V3KHEYSFOVOXM6QAVYXGI32LIR6GHPWBPN2YEINCVVZNLJURWAQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971077961 (67h38m7.53440095s) [noise=true , address=104.36.82.3:5773 , piece_id=V4LZ6Z3UENGZ6VPNZCVQPG364BRVIU2CXLMR2SFC7XFJUQOO3T6Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.971496241 (67h38m7.534819231s) [noise=true , address=104.36.82.10:6080 , piece_id=VJ2AIXVLL3I23RZ2NMA7XWQUVFMTFCZN5RN7TATN2JAZYKBMTHEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.972527742 (67h38m7.535850742s) [noise=true , address=104.36.82.52:4118 , piece_id=WTML7QNC22E4GKCBZODYKFVGFYNJAPGWLRLSKO6JGVYX27P6CXMQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973144505 (67h38m7.536467496s) [noise=true , address=104.36.82.56:4242 , piece_id=MKEDTELY5FDI7KP5S2Z3QX25MIM2P5FN5H6P4VAB7BW7NHXUDTXQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.973472837 (67h38m7.53679583s) [noise=true , address=104.36.82.53:4147 , piece_id=N2J7LL2EC35PVZ42A2ACCMS6T42Z4ICB27BOZYH2NMJJT7U2YCEA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977168261 (67h38m7.540491246s) [noise=true , address=104.36.82.21:6501 , piece_id=WDYNIJ3ECDPAWXPPSGH4EN7OFFNHJ5OHEISRKKX3YY57FRE5VSTA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977841468 (67h38m7.541164455s) [noise=true , address=104.36.82.49:4001 , piece_id=XEW5UNOXTU2IBCZDCFRUKMMO4JCKYSGLOO6EWTMZUQ7ORMBO3M7Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.977897711 (67h38m7.541220704s) [noise=true , address=104.36.82.45:6937 , piece_id=FEXDH4TWEXXR52GFRZWNMA7EHFERXHCVQZSQLOFUDGQ7YJK4OEUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978368788 (67h38m7.541691789s) [noise=true , address=104.36.82.55:4222 , piece_id=D7AZ26NCDHRQ4DT7VCWVXMVSYLLXWTULK7J62S3OZVBX6LTPXE2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978417585 (67h38m7.541740589s) [noise=true , address=104.36.82.13:6209 , piece_id=WNI3XQODGXBNS3QZD7DZ2MFWGDIX2KIVQEV72FYNTNJY4HFQITUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:31.978845444 (67h38m7.542168445s) [noise=true , address=104.36.82.54:4164 , piece_id=PBOP6ZENYGDSVPOQRDI32EVYWSDUVBK2ACTOXI6XA5XDK5SM66SQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.043968605 (67h38m7.607291609s) [noise=true , address=104.36.82.5:5859 , piece_id=XZ2WCC4A73OP5QJUMYL7R4GW6BE5DZRYSQMV4GM45HO2QIDTGD2Q, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051466593 (67h38m7.614789592s) [noise=true , address=79.127.220.101:6051 , piece_id=W5KS7RFQN77T3IKU66ZLA44OUSPUTYYQLANDDDT6NERRRFRYTPVQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: <nil> Jun 21 12:01:32.051490584 (67h38m7.614813583s) [seg_pos={0 0}, upload=54163]: Segment reached optimal threshold of 14 pieces. Jun 21 12:01:32.051824881 (67h38m7.615147879s) [noise=true , address=79.127.205.246:4266 , piece_id=FHHQH5B5F373I2RJP7I7JXB6RTULFN6USXAJSLC6J35OPEWHKVUA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wEmEB8Up7sKbBXuoZu9fx9e7cyuFDc8HVBt1g2x6ajDFj2Pnh): protocol: read tcp 159.223.143.39:44678->79.127.205.246:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051865623 (67h38m7.615188622s) [noise=true , address=79.127.205.245:4245 , piece_id=2XGQMNJOUSXWD75DNZLMLJBJSRG4JWGZSCWU6SSTDHSYO7JQHSAA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1S4wJRQEKESaiR7nPDqq8LZpqd2iYY9qTXF8SG8Kvf4sXsY8JH): protocol: read tcp 159.223.143.39:35280->79.127.205.245:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051906113 (67h38m7.615229111s) [noise=true , address=156.146.43.225:4440 , piece_id=W3GZH4YC4UW2T6X5TPZETWIAOMYLCRNVOMFGAWI5EX2LZYFD3XEQ, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:51204->156.146.43.225:4440: use of closed network connection Jun 21 12:01:32.051931091 (67h38m7.61525409s) [noise=true , address=79.127.205.233:5655 , piece_id=7TL63ITP3KRDFEXKZ2X2ZQOAYWK7X3NSGFIRQ5L7YK3JJCUC4RWA, seg_pos={0 0}, upload=54163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1anVAEEau5NDCw4VKRMxMUFKJ4DeiLeUACUmiRoqGs8gotTr1i): protocol: read tcp 159.223.143.39:34122->79.127.205.233:5655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 21 12:01:32.051949564 (67h38m7.615272564s) [seg_pos={0 0}, upload=54163]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 21 12:01:32.052014337 (67h38m7.615337336s) [upload=54163]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 21 12:01:32.052017771 (67h38m7.61534077s) [upload=54163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 21 12:01:32.105463539 (67h38m7.668786539s) [upload=54163]: Done uploading segments... Jun 21 12:01:32.105486328 (67h38m7.668809327s) [upload=54163]: Upload finished. err: <nil> Duration:* 397.601644ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 655.48563ms FAIL (2025-06-21 07:41:14 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 655.5271ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 698.735682ms FAIL (2025-06-21 07:40:58 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 698.756553ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 742.218814ms FAIL (2025-06-21 07:40:29 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 742.245962ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 641.268826ms FAIL (2025-06-21 07:39:16 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 641.344373ms core uplink us-select-1 us1-placement-20
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 253.95516ms FAIL (2025-06-21 07:39:01 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 253.983355ms edge authservice http bgp us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 684.795759ms FAIL (2025-06-21 07:38:39 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 684.817135ms core uplink us-select-1 us1-placement-20
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 140.526383ms FAIL (2025-06-21 07:38:16 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 140.573335ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 259.763234ms FAIL (2025-06-21 07:37:49 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 259.809067ms edge authservice http bgp us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 655.992052ms FAIL (2025-06-21 07:37:30 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 656.013527ms core uplink us-select-1 us1-placement-20
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 223.412239ms FAIL (2025-06-21 07:37:30 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 223.436162ms edge authservice http bgp us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 903.97947ms FAIL (2025-06-21 07:37:05 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 904.090029ms core uplink us-select-1 us1-placement-20
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 262.49917ms FAIL (2025-06-21 07:36:28 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 262.538034ms edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 165.100675ms FAIL (2025-06-21 07:36:08 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 165.14772ms edge authservice http bgp us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 1.160662525s FAIL (2025-06-21 07:36:01 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.160688383s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 641.110778ms FAIL (2025-06-21 07:35:35 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 641.140083ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 828.546102ms FAIL (2025-06-21 07:35:18 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 828.5836ms core uplink us-select-1 us1-placement-20
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 269.362443ms FAIL (2025-06-21 07:35:00 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 269.415259ms edge authservice http bgp us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 783.342559ms FAIL (2025-06-21 07:34:22 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 783.363297ms core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)