Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...XXX......................................................] duration: 435.671688ms FAIL (2025-06-25 07:02:56 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 435.700491ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....XXX.....................................................] duration: 505.054814ms FAIL (2025-06-25 07:02:31 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 505.093286ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....XXX....................................................] duration: 446.892894ms FAIL (2025-06-25 07:01:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 446.918841ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......XXX...................................................] duration: 714.49072ms FAIL (2025-06-25 07:00:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 714.538947ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......XXX..................................................] duration: 601.097984ms FAIL (2025-06-25 07:00:15 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 601.134689ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........XXX.................................................] duration: 450.548365ms FAIL (2025-06-25 06:59:49 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 450.583146ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........XXX................................................] duration: 411.348649ms FAIL (2025-06-25 06:58:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 411.383135ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........XXX...............................................] duration: 415.282912ms FAIL (2025-06-25 06:57:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 415.314069ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........XXX..............................................] duration: 462.008568ms FAIL (2025-06-25 06:57:17 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 462.037278ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............XXX.............................................] duration: 593.224375ms FAIL (2025-06-25 06:56:49 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 593.257562ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............XXX............................................] duration: 530.168962ms FAIL (2025-06-25 06:56:27 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 530.194443ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............XXX...........................................] duration: 426.17572ms FAIL (2025-06-25 06:55:56 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 426.207117ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............XXX..........................................] duration: 476.018696ms FAIL (2025-06-25 06:55:23 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 476.056237ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................XXX.........................................] duration: 436.071218ms FAIL (2025-06-25 06:54:51 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 436.10603ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................XXX........................................] duration: 465.582528ms FAIL (2025-06-25 06:54:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 465.613794ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................XXX.......................................] duration: 541.68671ms FAIL (2025-06-25 06:54:04 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 541.740687ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................XXX......................................] duration: 404.130105ms FAIL (2025-06-25 06:53:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 404.16128ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................XXX.....................................] duration: 434.49012ms FAIL (2025-06-25 06:52:02 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 434.519355ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................XXX....................................] duration: 417.505185ms FAIL (2025-06-25 06:51:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 417.647186ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................XXX...................................] duration: 473.036513ms FAIL (2025-06-25 06:50:33 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 473.059752ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................XXX..................................] duration: 505.594784ms FAIL (2025-06-25 06:50:03 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 505.663847ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................XXX.................................] duration: 481.288967ms FAIL (2025-06-25 06:49:43 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 481.386008ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................XXX................................] duration: 409.689861ms FAIL (2025-06-25 06:48:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 409.717722ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........................XXX...............................] duration: 505.113982ms FAIL (2025-06-25 06:48:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 505.141334ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........................XXX..............................] duration: 507.573574ms FAIL (2025-06-25 06:47:22 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 507.600189ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............................XXX.............................] duration: 519.602774ms FAIL (2025-06-25 06:46:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 519.657295ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............................XXX............................] duration: 465.947888ms FAIL (2025-06-25 06:45:38 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 465.978291ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............................XXX...........................] duration: 498.215396ms FAIL (2025-06-25 06:45:14 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 498.248737ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............................XXX..........................] duration: 590.542439ms FAIL (2025-06-25 06:44:27 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 590.568659ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................................XXX.........................] duration: 409.75826ms FAIL (2025-06-25 06:43:51 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 409.791732ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................................XXX........................] duration: 635.619589ms FAIL (2025-06-25 06:43:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 635.732881ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................................XXX.......................] duration: 399.835412ms FAIL (2025-06-25 06:41:58 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 399.867795ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................................XXX......................] duration: 443.396497ms FAIL (2025-06-25 06:41:25 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 443.427797ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................................XXX.....................] duration: 418.410421ms FAIL (2025-06-25 06:40:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 418.44262ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................................XXX....................] duration: 514.575291ms FAIL (2025-06-25 06:40:06 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 514.611056ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................................XXX...................] duration: 489.508294ms FAIL (2025-06-25 06:39:49 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 489.538677ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................................XXX..................] duration: 492.775007ms FAIL (2025-06-25 06:39:05 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 492.814317ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................................XXX.................] duration: 460.649315ms FAIL (2025-06-25 06:38:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 460.681121ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................XXX................] duration: 423.740372ms FAIL (2025-06-25 06:37:25 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 423.769298ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........................................XXX...............] duration: 758.279189ms FAIL (2025-06-25 06:36:12 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 758.330937ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........................................XXX..............] duration: 444.502342ms FAIL (2025-06-25 06:35:27 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 444.528577ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............................................XXX.............] duration: 550.389531ms FAIL (2025-06-25 06:34:14 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 550.430871ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............................................XXX............] duration: 478.302584ms FAIL (2025-06-25 06:33:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 478.346387ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............................................XXX...........] duration: 401.514177ms FAIL (2025-06-25 06:33:08 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 401.543521ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............................................XXX..........] duration: 419.113596ms FAIL (2025-06-25 06:31:55 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 419.14077ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................................................XXX.........] duration: 499.367018ms FAIL (2025-06-25 06:31:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 499.397597ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................................................XXX........] duration: 832.582077ms FAIL (2025-06-25 06:30:22 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 832.614141ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................................................XXX.......] duration: 884.449812ms FAIL (2025-06-25 06:29:16 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 884.477217ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................................................XXX......] duration: 961.223985ms FAIL (2025-06-25 06:28:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 961.258986ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................................................XXX.....] duration: 973.630313ms FAIL (2025-06-25 06:28:25 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.965579422 (45h13m33.89187412s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.966861056 (45h13m33.893155753s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.967637432 (45h13m33.893932129s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.968622153 (45h13m33.894916849s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969160940 (45h13m33.895455637s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.969722201 (45h13m33.896016898s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.971267136 (45h13m33.897561831s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.973975937 (45h13m33.900270634s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.977444569 (45h13m33.903739259s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.979966838 (45h13m33.906261532s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.985254518 (45h13m33.911549213s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.989690365 (45h13m33.915985038s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.992878963 (45h13m33.919173668s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:06.993873142 (45h13m33.920167832s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Uploading piece... Jun 25 06:25:07.081827309 (45h13m34.008121992s) [upload=36292]: Next returned nil segment Jun 25 06:25:07.081846644 (45h13m34.008141321s) [upload=36292]: Waiting for error group managing segments... Jun 25 06:25:07.091339199 (45h13m34.017633886s) [noise=true , address=104.36.82.9:4007 , piece_id=AXF6U5YQMTQ4FTC6N5FHQ5GF523FRWRPZI3J47CAU7WGBA22AOMQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.091620012 (45h13m34.017914693s) [noise=true , address=104.36.82.2:4730 , piece_id=C2FUFWWZ4M7FLYHP3YICZDY43XW5Q33WV5T5OJ3PAFFC665RB5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.092011563 (45h13m34.018306239s) [noise=true , address=104.36.82.12:4150 , piece_id=CIPRIXF6MTKYHPAIWHPHZURQSDOZKVB5Q33ODZLVVGODKJMA4BRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:07.098034961 (45h13m34.024329637s) [noise=true , address=104.36.82.7:4909 , piece_id=H5O7BDH2C4OZZNXQDZMHULFJZLFG3F3E3J64YFSOE6264LYQ3PRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: <nil> Jun 25 06:25:36.770254614 (45h14m3.696549312s) [noise=true , address=45.91.116.5:5297 , piece_id=JYJOZAPJLERLXR4FDQNZGPYVNUV5RBDX536ASHLZNFLVIURUK64A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:5297): protocol: write tcp 159.223.143.39:57038->45.91.116.5:5297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770344543 (45h14m3.696639233s) [noise=true , address=77.90.31.16:4735 , piece_id=WB2PINNS7NRWK7L7WALOGEP35W35DFMTDLEUSXBLRDBU6GMFC5JQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 159.223.143.39:50774->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770376688 (45h14m3.696671402s) [noise=true , address=45.91.116.9:5261 , piece_id=SR7WE2CJ55NL24TACF4D7SNJ5OPMV5HEXLNWN46UEZTIODTTDKBA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: rpc: dial tcp 45.91.116.9:5261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770509119 (45h14m3.696803819s) [noise=true , address=77.90.31.19:4842 , piece_id=IKRE5LN6EE3G7HKFYCKJJZOWNIT25QTC5RTVW2Y6UXIGJKAVC75A, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770535520 (45h14m3.69683021s) [noise=true , address=77.90.31.11:4553 , piece_id=ZHFZMAAWL4E5KFD2WLUZOKNHJOTOKROWP4YY3FMIJC5MXBMD5CRQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 159.223.143.39:34200->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770589717 (45h14m3.696884407s) [noise=true , address=45.91.116.10:4081 , piece_id=ZRX5JL2SDHZLHYCZUPCOMQAYJR4L2V3KFFZEUTWMZJXEIPAWGXQQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 159.223.143.39:54710->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770603698 (45h14m3.696898388s) [noise=true , address=77.90.31.15:4698 , piece_id=3XQAGT7DDKO4EU6W4IIBPPDOUGOOH6JYZPHFEGLNFEN4ZEF3N2MQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770668501 (45h14m3.69696319s) [noise=true , address=45.91.116.6:4975 , piece_id=NOYWU54CFIC6WR3PHPIBDMSAUYXYIOILRVZH3KYQNSQKPYCA6I4Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770681314 (45h14m3.696976002s) [noise=true , address=45.91.116.11:4119 , piece_id=JWDCHDU5567YZ3R4PLRFFN2IPLG7BOVZD45VPBVC57R6HVIHJ32Q, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:60876->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770735699 (45h14m3.697030391s) [noise=true , address=77.90.31.11:4555 , piece_id=GSGAU5KRRMWH2ZX5RHRYOBOH5XWD64YFF5J7ZIMSOTUH76WU33LA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 159.223.143.39:39044->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770761378 (45h14m3.697056067s) [noise=true , address=45.91.116.12:5227 , piece_id=GKX6OV5DVRVESJAM6M7MBQNLULWXGQY5HLDZIVM27UU3HINIB4HA, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:5227): protocol: write tcp 159.223.143.39:52044->45.91.116.12:5227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770771668 (45h14m3.697066369s) [noise=true , address=45.91.116.7:4009 , piece_id=INLG4TJ5WAQ4AK23HM4S56JOYLU3QTKPMMCEL2OACT5P3VG446XQ, seg_pos={0 0}, upload=36292]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 159.223.143.39:50972->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 06:25:36.770793225 (45h14m3.697087925s) [seg_pos={0 0}, upload=36292]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jun 25 06:25:36.807855113 (45h14m3.734149801s) [upload=36292]: Done uploading segments... Jun 25 06:25:36.807886310 (45h14m3.734199715s) [upload=36292]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 973.659416ms core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)