Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure

Orbiter Check History

Showing 50 of 152001 reports (Page 2501 of 3041)
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
6 out of 60 is failed (threshold=3) [....X.............XX...........X...X...X....................]
duration: 
445.724388ms
FAIL (2025-07-28 01:00:32 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
0}, upload=25651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 159.223.143.39:39890->104.36.82.7:4909: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:44:18.975782256 (26h52m34.756715638s) [seg_pos={0 0}, upload=25651]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 28 00:44:18.975871874 (26h52m34.756805277s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 28 00:44:18.975878645 (26h52m34.756812034s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 28 00:44:19.019856339 (26h52m34.800789708s) [upload=25651]: Done uploading segments...
Jul 28 00:44:19.019879905 (26h52m34.800813283s) [upload=25651]: Upload finished. err: <nil>
Jul 28 00:45:02.257192553 (26h53m18.03812594s) [upload=25659]: Starting upload
Jul 28 00:45:02.257306836 (26h53m18.038240216s) [upload=25659]: Done starting upload
Jul 28 00:45:02.257434360 (26h53m18.038367739s) [upload=25659]: Uploading segments...
Jul 28 00:45:02.257858888 (26h53m18.038792251s) [upload=25659]: Got next segment. Inline: false
Jul 28 00:45:02.257951856 (26h53m18.038885235s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 28 00:45:02.257959915 (26h53m18.038893293s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 28 00:45:02.293048419 (26h53m18.073981806s) [seg_pos={0 0}, upload=25659]: Begin upload segment...
Jul 28 00:45:02.293125122 (26h53m18.074058491s) [seg_pos={0 0}, upload=25659]: Done begin upload segment.
Jul 28 00:45:02.293171689 (26h53m18.074105073s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293724212 (26h53m18.074657618s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293744577 (26h53m18.074677964s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294292781 (26h53m18.075226212s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294343706 (26h53m18.075277094s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294756878 (26h53m18.075690269s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294891918 (26h53m18.075825306s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295221121 (26h53m18.076154513s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295558583 (26h53m18.076491975s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295604310 (26h53m18.0765377s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295991730 (26h53m18.07692512s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.296088221 (26h53m18.077021615s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.301695563 (26h53m18.082628952s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302203775 (26h53m18.083137164s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302629001 (26h53m18.083562389s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302995157 (26h53m18.083928524s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.418208178 (26h53m18.19914155s) [upload=25659]: Next returned nil segment
Jul 28 00:45:02.418226832 (26h53m18.19916021s) [upload=25659]: Waiting for error group managing segments...
Jul 28 00:45:02.424117363 (26h53m18.205050731s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.427234519 (26h53m18.208167896s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.430114806 (26h53m18.211048185s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.435624120 (26h53m18.216557505s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436119655 (26h53m18.217053027s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436405218 (26h53m18.217338582s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436887494 (26h53m18.217820863s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436998726 (26h53m18.217932114s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437527079 (26h53m18.218460459s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437625274 (26h53m18.218558642s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439469614 (26h53m18.220402995s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439557143 (26h53m18.220490506s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:32.221331076 (26h53m48.002264449s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: write tcp 159.223.143.39:33378->104.36.82.16:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221411781 (26h53m48.002345162s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221457184 (26h53m48.002390547s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: rpc: dial tcp 104.36.82.13:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221473904 (26h53m48.002407294s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:49546->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221516309 (26h53m48.002449671s) [seg_pos={0 0}, upload=25659]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul 28 00:45:32.251981717 (26h53m48.032915086s) [upload=25659]: Done uploading segments...
Jul 28 00:45:32.252005327 (26h53m48.032938704s) [upload=25659]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 445.765577ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip: 
8 out of 60 is failed (threshold=3) [..XX.......X...........X...................X....XX........X.]
duration: 
440.452912ms
FAIL (2025-07-28 01:00:20 UTC) 
orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 440.488277ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip: 
8 out of 60 is failed (threshold=3) [...XX.......X...........X...................X....XX........X]
duration: 
30.029732497s
FAIL (2025-07-28 00:59:55 UTC) 
orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 30.029846033s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)
antiblip: 
53 out of 60 is failed (threshold=3) [......XXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration: 
1m0.032051637s
FAIL (2025-07-28 00:59:42 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 2)
======LOGS======
oading piece...
Jul 28 00:58:42.920761030 (27h6m58.701694414s) [noise=true , address=104.36.82.13:4190    , piece_id=SQGARLKSSHKFL5FH4GEQMLC7PY4HJOCQBDN3DB2MFX5HFEPVAVQQ, seg_pos={0 0}, upload=25871]: Uploading piece...
Jul 28 00:58:42.921108949 (27h6m58.702042334s) [noise=true , address=104.36.82.10:4081    , piece_id=Q7O6WWY7HFJHLDRNO6CYG7YP4QW6X4UJY7ZUHSSZT2BO7D2DUZQQ, seg_pos={0 0}, upload=25871]: Uploading piece...
Jul 28 00:58:42.921466142 (27h6m58.702399527s) [noise=true , address=104.36.82.54:4171    , piece_id=USSDKDKIFVTBZRFU2PN2K7T6LW46PKRFTH3ELCMAP7UX4IQHBA3Q, seg_pos={0 0}, upload=25871]: Uploading piece...
Jul 28 00:58:42.921826467 (27h6m58.702759854s) [noise=true , address=104.36.82.11:4134    , piece_id=DW7HUQDX322UV73VZY7GRU3DPCY7X5Y22RKXJAC326SUBD5H4NAA, seg_pos={0 0}, upload=25871]: Uploading piece...
Jul 28 00:58:42.922187790 (27h6m58.703121176s) [noise=true , address=79.127.205.233:4656  , piece_id=6XI43T4265WABRLLPTI6Z5QYPPJBALU3TZFIZYMEIMIWF6KWB5YA, seg_pos={0 0}, upload=25871]: Uploading piece...
Jul 28 00:58:42.922557597 (27h6m58.703490987s) [noise=true , address=104.36.82.15:4291    , piece_id=YSQKRLRGUWA7ZBHSEYGDRFSZJ6QM6V2C67QKZPZIML2KT6GWIMZQ, seg_pos={0 0}, upload=25871]: Uploading piece...
Jul 28 00:58:42.923306245 (27h6m58.704239631s) [noise=true , address=104.36.82.3:4772     , piece_id=FRFZEGOSNJGD4V44NHXOXP2SHH7XUMVOXLRVN4PT4R5KUVXSOVLQ, seg_pos={0 0}, upload=25871]: Uploading piece...
Jul 28 00:58:42.923668231 (27h6m58.704601613s) [noise=true , address=104.36.82.52:4086    , piece_id=GYCRR6TLR4KV2JALSOGUGU5M4QXBRCGBPAVDNMCWDLJAAUJMSFBA, seg_pos={0 0}, upload=25871]: Uploading piece...
Jul 28 00:58:42.924634019 (27h6m58.705567412s) [noise=true , address=79.127.201.215:4357  , piece_id=BJLXTRK2URT7YCY2L2R7J6IJ7A2IT5D7C2PVIX2SU3JOY5J3GX7Q, seg_pos={0 0}, upload=25871]: Uploading piece...
Jul 28 00:58:43.165125988 (27h6m58.946059365s) [upload=25871]: Next returned nil segment
Jul 28 00:58:43.165148241 (27h6m58.946081618s) [upload=25871]: Waiting for error group managing segments...
Jul 28 00:58:43.349855179 (27h6m59.130788558s) [noise=true , address=79.127.201.215:4357  , piece_id=BJLXTRK2URT7YCY2L2R7J6IJ7A2IT5D7C2PVIX2SU3JOY5J3GX7Q, seg_pos={0 0}, upload=25871]: Done uploading piece. err: <nil>
Jul 28 00:58:43.527358374 (27h6m59.308291742s) [noise=true , address=79.127.205.233:4656  , piece_id=6XI43T4265WABRLLPTI6Z5QYPPJBALU3TZFIZYMEIMIWF6KWB5YA, seg_pos={0 0}, upload=25871]: Done uploading piece. err: <nil>
Jul 28 00:59:42.847160147 (27h7m58.628093525s) [noise=true , address=104.36.82.12:4153    , piece_id=ZORXMG4BWY36V6MOWNULRUANS4LN2C4HHTEE7UAPUT4ZT3RYDSUA, seg_pos={0 0}, upload=25871]: Done uploading piece. err: ecclient: upload failed (node:124myE1wFhvjA5paJy5tr2hRe8jZCoEdbJLdBwvss1ZnT4GvtZv, address:104.36.82.12:4153): protocol: rpc: dial tcp 104.36.82.12:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:59:42.847251351 (27h7m58.628184727s) [noise=true , address=104.36.82.7:4930     , piece_id=SLDMWVEF4HK2SMRGO5EZCL3MYPHYK6QXK35DGOWKJSQRBDXJ3ANQ, seg_pos={0 0}, upload=25871]: Done uploading piece. err: ecclient: upload failed (node:1W215rQvfvgCZHym7547eECCCfSqEe2pe3R8jWRusjzNCp5Jac, address:104.36.82.7:4930): protocol: rpc: dial tcp 104.36.82.7:4930: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:59:42.847328323 (27h7m58.628261699s) [noise=true , address=104.36.82.4:4824     , piece_id=CALEDQ6YSGABNEBN65ISSFQARUHONRXL72ZYS2TYWQ4RF2455MUQ, seg_pos={0 0}, upload=25871]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:104.36.82.4:4824): protocol: write tcp 159.223.143.39:47380->104.36.82.4:4824: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:59:42.847403749 (27h7m58.628337126s) [noise=true , address=104.36.82.53:4138    , piece_id=6RSXJ4YU3QTDFVJISUXT3EU7F3VJ2H53DNWNFM2TGRARVYTIDCEA, seg_pos={0 0}, upload=25871]: Done uploading piece. err: ecclient: upload failed (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3, address:104.36.82.53:4138): protocol: rpc: dial tcp 104.36.82.53:4138: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:59:42.847474691 (27h7m58.628408068s) [noise=true , address=104.36.82.13:4190    , piece_id=SQGARLKSSHKFL5FH4GEQMLC7PY4HJOCQBDN3DB2MFX5HFEPVAVQQ, seg_pos={0 0}, upload=25871]: Done uploading piece. err: ecclient: upload failed (node:1eNJzs5Dc7Nv7imoQXZDp2mEVWjD8o4viGVkFvuAeZ1arynxu4, address:104.36.82.13:4190): protocol: rpc: dial tcp 104.36.82.13:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:59:42.847541225 (27h7m58.628474591s) [noise=true , address=104.36.82.52:4086    , piece_id=GYCRR6TLR4KV2JALSOGUGU5M4QXBRCGBPAVDNMCWDLJAAUJMSFBA, seg_pos={0 0}, upload=25871]: Done uploading piece. err: ecclient: upload failed (node:1SqSapYVaJiXrqqGfD9a1XFgf6Y8eGJXmudUmLhX7yjmYXBWTm, address:104.36.82.52:4086): protocol: rpc: dial tcp 104.36.82.52:4086: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:59:42.847548341 (27h7m58.628481706s) [noise=true , address=104.36.82.41:4790    , piece_id=Y4574NUBNS2O6ZFNZX57TF6WRWC2Q6MLGQ4TQC5IXQG2LNULGPNQ, seg_pos={0 0}, upload=25871]: Done uploading piece. err: ecclient: upload failed (node:12kuSMof8EbvsaMCdVxKWjgTT5TfLJnTAeHUf8DKNC3UGQn5q7T, address:104.36.82.41:4790): protocol: rpc: dial tcp 104.36.82.41:4790: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:59:42.847605279 (27h7m58.628538643s) [noise=true , address=104.36.82.11:4141    , piece_id=F4IAW2QGYVCECCBOVX6FZRBFABWUU6YQI5ANCW4RRNXZ6ORKLG3Q, seg_pos={0 0}, upload=25871]: Done uploading piece. err: ecclient: upload failed (node:129zfpWSguxcUH9723ivdoL39WgpjiTxA189mYbzsGAGVjFcXLL, address:104.36.82.11:4141): protocol: rpc: dial tcp 104.36.82.11:4141: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:59:42.847615401 (27h7m58.628548767s) [noise=true , address=104.36.82.3:4772     , piece_id=FRFZEGOSNJGD4V44NHXOXP2SHH7XUMVOXLRVN4PT4R5KUVXSOVLQ, seg_pos={0 0}, upload=25871]: Done uploading piece. err: ecclient: upload failed (node:1BP2yptVJkJzCKenuKzMZ5sBbVbqXcaoVviwKwrnbxodo1pR5w, address:104.36.82.3:4772): protocol: rpc: dial tcp 104.36.82.3:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:59:42.847624040 (27h7m58.628557407s) [noise=true , address=104.36.82.42:4829    , piece_id=7OQ4LKGG4BM3EG77YBGUQSLGYJ4KGSDZ6JX35CPWS6OBFBBZ6MRQ, seg_pos={0 0}, upload=25871]: Done uploading piece. err: ecclient: upload failed (node:1F5T8Vt1x4nXFaSenvGBCYjxhRL1H1sxRvzLjW3CqzwDVhq39F, address:104.36.82.42:4829): protocol: rpc: dial tcp 104.36.82.42:4829: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:59:42.847631838 (27h7m58.628565207s) [noise=true , address=104.36.82.15:4291    , piece_id=YSQKRLRGUWA7ZBHSEYGDRFSZJ6QM6V2C67QKZPZIML2KT6GWIMZQ, seg_pos={0 0}, upload=25871]: Done uploading piece. err: ecclient: upload failed (node:125JwnPDmKBNm3TybPWrzRbpzo5pnLei38joZeetvDG3wU32ZNb, address:104.36.82.15:4291): protocol: rpc: dial tcp 104.36.82.15:4291: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:59:42.847695056 (27h7m58.62862842s) [noise=true , address=104.36.82.22:4536    , piece_id=TQHD6ZR2ONFHG4RKD7FNCYVBMAPZ5IZAFLLP6HHSLP6RQCUNGHOQ, seg_pos={0 0}, upload=25871]: Done uploading piece. err: ecclient: upload failed (node:12C2G1QGN37Wu55dBAxLiYWYAXmGapzjWUZUCKgCUbcjXcaRFmt, address:104.36.82.22:4536): protocol: rpc: dial tcp 104.36.82.22:4536: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:59:42.847710617 (27h7m58.628643983s) [noise=true , address=104.36.82.54:4171    , piece_id=USSDKDKIFVTBZRFU2PN2K7T6LW46PKRFTH3ELCMAP7UX4IQHBA3Q, seg_pos={0 0}, upload=25871]: Done uploading piece. err: ecclient: upload failed (node:125SMhDsEZruNu6LkmQn2M7R6WzvXTdk86DRsKf522u8BNiANq3, address:104.36.82.54:4171): protocol: rpc: dial tcp 104.36.82.54:4171: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:59:42.847767590 (27h7m58.628700951s) [noise=true , address=104.36.82.10:4081    , piece_id=Q7O6WWY7HFJHLDRNO6CYG7YP4QW6X4UJY7ZUHSSZT2BO7D2DUZQQ, seg_pos={0 0}, upload=25871]: Done uploading piece. err: ecclient: upload failed (node:12t9rGfCQ7bLkNnZzaPVgkE2Y9rXsiFp1XGdeECzv9M1jk8Qqmp, address:104.36.82.10:4081): protocol: rpc: dial tcp 104.36.82.10:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:59:42.847782245 (27h7m58.628715611s) [noise=true , address=104.36.82.55:4203    , piece_id=3ZRDAJK5JLFUQJXP3E6F3FPPUTSVD7RM4MVR7F7SI7IIQ4F7FR4Q, seg_pos={0 0}, upload=25871]: Done uploading piece. err: ecclient: upload failed (node:12ZewP4Cfpax2814Hqbtfvp4PzcDX455Gv7WwSTSwt1NTMXzYhU, address:104.36.82.55:4203): protocol: rpc: dial tcp 104.36.82.55:4203: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:59:42.847792162 (27h7m58.62872553s) [noise=true , address=104.36.82.11:4134    , piece_id=DW7HUQDX322UV73VZY7GRU3DPCY7X5Y22RKXJAC326SUBD5H4NAA, seg_pos={0 0}, upload=25871]: Done uploading piece. err: ecclient: upload failed (node:12R3Z5AgkBmWXMv5XgRyfcyqkG2gmyJs9ogT24kcLy6nuB9bzUG, address:104.36.82.11:4134): protocol: write tcp 159.223.143.39:36042->104.36.82.11:4134: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:59:42.847821073 (27h7m58.628754448s) [seg_pos={0 0}, upload=25871]: Done waiting for segment. successful: 2 optimal: 14 errs: <nil>
Jul 28 00:59:42.876053610 (27h7m58.656986988s) [upload=25871]: Done uploading segments...
Jul 28 00:59:42.876083385 (27h7m58.657016767s) [upload=25871]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 2)
Duration: 1m0.032072444s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
6 out of 60 is failed (threshold=3) [.....X.............XX...........X...X...X...................]
duration: 
411.330743ms
FAIL (2025-07-28 00:59:28 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
0}, upload=25651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 159.223.143.39:39890->104.36.82.7:4909: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:44:18.975782256 (26h52m34.756715638s) [seg_pos={0 0}, upload=25651]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 28 00:44:18.975871874 (26h52m34.756805277s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 28 00:44:18.975878645 (26h52m34.756812034s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 28 00:44:19.019856339 (26h52m34.800789708s) [upload=25651]: Done uploading segments...
Jul 28 00:44:19.019879905 (26h52m34.800813283s) [upload=25651]: Upload finished. err: <nil>
Jul 28 00:45:02.257192553 (26h53m18.03812594s) [upload=25659]: Starting upload
Jul 28 00:45:02.257306836 (26h53m18.038240216s) [upload=25659]: Done starting upload
Jul 28 00:45:02.257434360 (26h53m18.038367739s) [upload=25659]: Uploading segments...
Jul 28 00:45:02.257858888 (26h53m18.038792251s) [upload=25659]: Got next segment. Inline: false
Jul 28 00:45:02.257951856 (26h53m18.038885235s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 28 00:45:02.257959915 (26h53m18.038893293s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 28 00:45:02.293048419 (26h53m18.073981806s) [seg_pos={0 0}, upload=25659]: Begin upload segment...
Jul 28 00:45:02.293125122 (26h53m18.074058491s) [seg_pos={0 0}, upload=25659]: Done begin upload segment.
Jul 28 00:45:02.293171689 (26h53m18.074105073s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293724212 (26h53m18.074657618s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293744577 (26h53m18.074677964s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294292781 (26h53m18.075226212s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294343706 (26h53m18.075277094s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294756878 (26h53m18.075690269s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294891918 (26h53m18.075825306s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295221121 (26h53m18.076154513s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295558583 (26h53m18.076491975s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295604310 (26h53m18.0765377s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295991730 (26h53m18.07692512s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.296088221 (26h53m18.077021615s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.301695563 (26h53m18.082628952s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302203775 (26h53m18.083137164s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302629001 (26h53m18.083562389s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302995157 (26h53m18.083928524s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.418208178 (26h53m18.19914155s) [upload=25659]: Next returned nil segment
Jul 28 00:45:02.418226832 (26h53m18.19916021s) [upload=25659]: Waiting for error group managing segments...
Jul 28 00:45:02.424117363 (26h53m18.205050731s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.427234519 (26h53m18.208167896s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.430114806 (26h53m18.211048185s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.435624120 (26h53m18.216557505s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436119655 (26h53m18.217053027s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436405218 (26h53m18.217338582s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436887494 (26h53m18.217820863s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436998726 (26h53m18.217932114s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437527079 (26h53m18.218460459s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437625274 (26h53m18.218558642s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439469614 (26h53m18.220402995s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439557143 (26h53m18.220490506s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:32.221331076 (26h53m48.002264449s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: write tcp 159.223.143.39:33378->104.36.82.16:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221411781 (26h53m48.002345162s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221457184 (26h53m48.002390547s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: rpc: dial tcp 104.36.82.13:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221473904 (26h53m48.002407294s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:49546->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221516309 (26h53m48.002449671s) [seg_pos={0 0}, upload=25659]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul 28 00:45:32.251981717 (26h53m48.032915086s) [upload=25659]: Done uploading segments...
Jul 28 00:45:32.252005327 (26h53m48.032938704s) [upload=25659]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 411.383904ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)
antiblip: 
52 out of 60 is failed (threshold=3) [.......XXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration: 
1m0.037830783s
FAIL (2025-07-28 00:58:42 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 2)
======LOGS======
NWWQ4AC7MAG4HCZF4ETCFDQDSDAUEMLYUTQ7GS5XFAHXEQ, seg_pos={0 0}, upload=25856]: Uploading piece...
Jul 28 00:57:42.819135130 (27h5m58.600068571s) [noise=true , address=104.36.82.21:4485    , piece_id=YIB6ZFJWDD5TZE56JI3QEP4BGIWQEKGHAXZ3WYISEY4BXH6IEQPQ, seg_pos={0 0}, upload=25856]: Uploading piece...
Jul 28 00:57:42.819625392 (27h5m58.600558778s) [noise=true , address=104.36.82.11:4119    , piece_id=ATPNBB4IWTBU4UYY2DL2DESIZ3IPYE7UAZXK5BXRC3FJOPCRSCWQ, seg_pos={0 0}, upload=25856]: Uploading piece...
Jul 28 00:57:42.821734111 (27h5m58.602667496s) [noise=true , address=79.127.205.235:4680  , piece_id=3TPMEOXB7WNVRXY5ZEQO3KVT5SAXWTG7P3LLGULY5T3B6DNICBOQ, seg_pos={0 0}, upload=25856]: Uploading piece...
Jul 28 00:57:42.823801648 (27h5m58.604735032s) [noise=true , address=104.36.82.15:4267    , piece_id=6N7MD7B66U24JRTROPQNZE75UQQEDFAHTKIB4O7FUIBA2BFNFLAA, seg_pos={0 0}, upload=25856]: Uploading piece...
Jul 28 00:57:42.824386523 (27h5m58.605319908s) [noise=true , address=79.127.201.215:4357  , piece_id=POESCI4XHELOAUKEXLPYDR2MQR3UISKIF2JPNRDXRMYN3FHV3N2Q, seg_pos={0 0}, upload=25856]: Uploading piece...
Jul 28 00:57:42.824985832 (27h5m58.605919217s) [noise=true , address=104.36.82.42:4798    , piece_id=TQN6RNELBVDSBVRYSR3Y5WAVP27FLR37PVVKN6V52NNEUMNN6BWA, seg_pos={0 0}, upload=25856]: Uploading piece...
Jul 28 00:57:42.833133140 (27h5m58.614066525s) [noise=true , address=104.36.82.56:4255    , piece_id=6FEVJLP3F6Y7OTRKZCDXR5YJI7JFRN47HZGXJWZBH4KDFS2GQGZA, seg_pos={0 0}, upload=25856]: Uploading piece...
Jul 28 00:57:42.835812732 (27h5m58.616746117s) [noise=true , address=104.36.82.47:4539    , piece_id=W3JP5SKNIAGMQXNIPH56Y44CLZAXBWXHTQHITUP53DP5PHTRWRCQ, seg_pos={0 0}, upload=25856]: Uploading piece...
Jul 28 00:57:43.033255691 (27h5m58.814189058s) [upload=25856]: Next returned nil segment
Jul 28 00:57:43.033702844 (27h5m58.814636225s) [upload=25856]: Waiting for error group managing segments...
Jul 28 00:57:43.650136813 (27h5m59.431070188s) [noise=true , address=79.127.205.235:4680  , piece_id=3TPMEOXB7WNVRXY5ZEQO3KVT5SAXWTG7P3LLGULY5T3B6DNICBOQ, seg_pos={0 0}, upload=25856]: Done uploading piece. err: <nil>
Jul 28 00:57:47.865747052 (27h6m3.646680436s) [noise=true , address=79.127.201.215:4357  , piece_id=POESCI4XHELOAUKEXLPYDR2MQR3UISKIF2JPNRDXRMYN3FHV3N2Q, seg_pos={0 0}, upload=25856]: Done uploading piece. err: <nil>
Jul 28 00:58:42.741963645 (27h6m58.522897013s) [noise=true , address=104.36.82.13:4216    , piece_id=3YCG63G3QM2ECWQJXNS54BOUEUT2FML4RQAZ3X2GF5FXJNLXX5LA, seg_pos={0 0}, upload=25856]: Done uploading piece. err: ecclient: upload failed (node:12AgazvjebAo7K4NdQ7zsKkETiNQNfiykHcBDcmXej6eipwWpFu, address:104.36.82.13:4216): protocol: rpc: dial tcp 104.36.82.13:4216: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:58:42.742043696 (27h6m58.522977075s) [noise=true , address=104.36.82.12:4179    , piece_id=ILYARITK7TKR74OZV2WRM2UIUKX2WFR2UMEKK77VBL2A5T7IHEMQ, seg_pos={0 0}, upload=25856]: Done uploading piece. err: ecclient: upload failed (node:129pP9f3rFfTvP6MhG894N7WHuJRZP8aWhzJf1zBwZ76wbzVbZa, address:104.36.82.12:4179): protocol: write tcp 159.223.143.39:42242->104.36.82.12:4179: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:58:42.742116630 (27h6m58.523049997s) [noise=true , address=104.36.82.52:4119    , piece_id=CQDSAQECIDS2D4F5W5HQL5PRHEQQV3L7NKOX5NJ22UGXIDXQXESQ, seg_pos={0 0}, upload=25856]: Done uploading piece. err: ecclient: upload failed (node:12mTLjL6Y6VVjqztwVXYif4v7tyftjLafztLUibajksMHCFraFG, address:104.36.82.52:4119): protocol: rpc: dial tcp 104.36.82.52:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:58:42.742183022 (27h6m58.523116388s) [noise=true , address=104.36.82.54:4190    , piece_id=74HXUJCRX7L7YQFUMKS4LBQY6K5XTF6TCDH2KVPH24NXU3VBBHLQ, seg_pos={0 0}, upload=25856]: Done uploading piece. err: ecclient: upload failed (node:1eKHcxWuBbQAGX44kVrn9cfEhEcRpmQfQegcoMGjr92pfM1VVF, address:104.36.82.54:4190): protocol: rpc: dial tcp 104.36.82.54:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:58:42.742247198 (27h6m58.523180575s) [noise=true , address=104.36.82.56:4262    , piece_id=SNHFX2NWWQ4AC7MAG4HCZF4ETCFDQDSDAUEMLYUTQ7GS5XFAHXEQ, seg_pos={0 0}, upload=25856]: Done uploading piece. err: ecclient: upload failed (node:1DY2kpdCo8EDSs1LHL6qBFo6qjnsb7FL1QDekMV5qaWB5jxeMz, address:104.36.82.56:4262): protocol: rpc: dial tcp 104.36.82.56:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:58:42.742313962 (27h6m58.523247346s) [noise=true , address=104.36.82.21:4485    , piece_id=YIB6ZFJWDD5TZE56JI3QEP4BGIWQEKGHAXZ3WYISEY4BXH6IEQPQ, seg_pos={0 0}, upload=25856]: Done uploading piece. err: ecclient: upload failed (node:1wmohWyFaMkNm8KBAZprmZ6RuiH6UcB8KRNght2HGpV3Ucnnq9, address:104.36.82.21:4485): protocol: rpc: dial tcp 104.36.82.21:4485: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:58:42.742378783 (27h6m58.523312164s) [noise=true , address=104.36.82.47:4539    , piece_id=W3JP5SKNIAGMQXNIPH56Y44CLZAXBWXHTQHITUP53DP5PHTRWRCQ, seg_pos={0 0}, upload=25856]: Done uploading piece. err: ecclient: upload failed (node:126g75XgAyFYcgnaibseqoGiz7cK6z3n6gPMcu9TrdjuZUYEHnZ, address:104.36.82.47:4539): protocol: write tcp 159.223.143.39:41666->104.36.82.47:4539: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:58:42.742471143 (27h6m58.523404507s) [noise=true , address=104.36.82.50:4043    , piece_id=6SBROYB5O7BHPA4JFJKSTJXP3LEQUONWEAMAVKKJGSUTSRLOTATQ, seg_pos={0 0}, upload=25856]: Done uploading piece. err: ecclient: upload failed (node:12f97TAyWcE4YrxDQCeycC8tnuetQiagu2MV4U5o8ARLq8jGKTX, address:104.36.82.50:4043): protocol: rpc: dial tcp 104.36.82.50:4043: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:58:42.742535472 (27h6m58.523468834s) [noise=true , address=104.36.82.20:4462    , piece_id=AJHS2KKDOTKJ6PJUFAOIWUOWXCCY6KLE3PUODNKKJCPZCZNLOZ3Q, seg_pos={0 0}, upload=25856]: Done uploading piece. err: ecclient: upload failed (node:15y4L1tWUv5vFniK5ugNHNYa22q7a7hC5Uf9pxkN4AuPBodcgY, address:104.36.82.20:4462): protocol: write tcp 159.223.143.39:48568->104.36.82.20:4462: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:58:42.742578863 (27h6m58.523512231s) [noise=true , address=104.36.82.42:4798    , piece_id=TQN6RNELBVDSBVRYSR3Y5WAVP27FLR37PVVKN6V52NNEUMNN6BWA, seg_pos={0 0}, upload=25856]: Done uploading piece. err: ecclient: upload failed (node:17Ski7oMzbGroPGgWRrbUiqC1NiLPBBsvYuU4pCZ53xhNKkeBZ, address:104.36.82.42:4798): protocol: rpc: dial tcp 104.36.82.42:4798: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:58:42.742599520 (27h6m58.523532883s) [noise=true , address=104.36.82.11:4119    , piece_id=ATPNBB4IWTBU4UYY2DL2DESIZ3IPYE7UAZXK5BXRC3FJOPCRSCWQ, seg_pos={0 0}, upload=25856]: Done uploading piece. err: ecclient: upload failed (node:121xNAnkm9HZXUszp26UmWTsuhp3igyeDVea6gSCdtrQUQDVHpW, address:104.36.82.11:4119): protocol: rpc: dial tcp 104.36.82.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:58:42.742662839 (27h6m58.523596214s) [noise=true , address=104.36.82.46:4484    , piece_id=GYEB2DHE47KFE7VVONKSUQOHFDFN37GI5ZPWZ2255D35CESNQNQA, seg_pos={0 0}, upload=25856]: Done uploading piece. err: ecclient: upload failed (node:1a5S5Ar7jNbK9MESghbWfmEgBPNSMN2tEnZeeG1kKzeX2dnSX3, address:104.36.82.46:4484): protocol: rpc: dial tcp 104.36.82.46:4484: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:58:42.742686507 (27h6m58.523619884s) [noise=true , address=104.36.82.15:4267    , piece_id=6N7MD7B66U24JRTROPQNZE75UQQEDFAHTKIB4O7FUIBA2BFNFLAA, seg_pos={0 0}, upload=25856]: Done uploading piece. err: ecclient: upload failed (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7, address:104.36.82.15:4267): protocol: rpc: dial tcp 104.36.82.15:4267: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:58:42.742738043 (27h6m58.523671418s) [noise=true , address=104.36.82.10:4088    , piece_id=7M7WVLGD4AJM7K3DDZVMBSATIB5AW55IGJNLPKTZ6OHNRSNDD2HA, seg_pos={0 0}, upload=25856]: Done uploading piece. err: ecclient: upload failed (node:12QbJar7xfiCubCHQ4hTTG4UYNVrxQig1mpvBKeXoEBUzbCHsJT, address:104.36.82.10:4088): protocol: write tcp 159.223.143.39:52762->104.36.82.10:4088: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:58:42.742767358 (27h6m58.523700727s) [noise=true , address=104.36.82.43:4868    , piece_id=N3WPE75QOGT6IWHB52M3U52MIL4SZ4PY37RC4K2D6GFDMDOJWZRA, seg_pos={0 0}, upload=25856]: Done uploading piece. err: ecclient: upload failed (node:124LJtYshPwu1uZJwVyjrjUMk3htyVnp9ntnZWSUb7X3bNkwU6t, address:104.36.82.43:4868): protocol: rpc: dial tcp 104.36.82.43:4868: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:58:42.742787924 (27h6m58.523721289s) [noise=true , address=104.36.82.56:4255    , piece_id=6FEVJLP3F6Y7OTRKZCDXR5YJI7JFRN47HZGXJWZBH4KDFS2GQGZA, seg_pos={0 0}, upload=25856]: Done uploading piece. err: ecclient: upload failed (node:12S6rwimXBvaEb4KneAwL5H3t5zTmvGVhyJSWCTU1u17r4RWJta, address:104.36.82.56:4255): protocol: write tcp 159.223.143.39:56710->104.36.82.56:4255: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:58:42.742834534 (27h6m58.523767909s) [seg_pos={0 0}, upload=25856]: Done waiting for segment. successful: 2 optimal: 14 errs: <nil>
Jul 28 00:58:42.778029951 (27h6m58.558963327s) [upload=25856]: Done uploading segments...
Jul 28 00:58:42.778058914 (27h6m58.558992293s) [upload=25856]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 2)
Duration: 1m0.037858095s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip: 
8 out of 60 is failed (threshold=3) [X...XX.......X...........X...................X....XX........]
duration: 
395.930143ms
FAIL (2025-07-28 00:58:35 UTC) 
orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 395.954645ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
6 out of 60 is failed (threshold=3) [......X.............XX...........X...X...X..................]
duration: 
371.219237ms
FAIL (2025-07-28 00:58:17 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
0}, upload=25651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 159.223.143.39:39890->104.36.82.7:4909: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:44:18.975782256 (26h52m34.756715638s) [seg_pos={0 0}, upload=25651]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 28 00:44:18.975871874 (26h52m34.756805277s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 28 00:44:18.975878645 (26h52m34.756812034s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 28 00:44:19.019856339 (26h52m34.800789708s) [upload=25651]: Done uploading segments...
Jul 28 00:44:19.019879905 (26h52m34.800813283s) [upload=25651]: Upload finished. err: <nil>
Jul 28 00:45:02.257192553 (26h53m18.03812594s) [upload=25659]: Starting upload
Jul 28 00:45:02.257306836 (26h53m18.038240216s) [upload=25659]: Done starting upload
Jul 28 00:45:02.257434360 (26h53m18.038367739s) [upload=25659]: Uploading segments...
Jul 28 00:45:02.257858888 (26h53m18.038792251s) [upload=25659]: Got next segment. Inline: false
Jul 28 00:45:02.257951856 (26h53m18.038885235s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 28 00:45:02.257959915 (26h53m18.038893293s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 28 00:45:02.293048419 (26h53m18.073981806s) [seg_pos={0 0}, upload=25659]: Begin upload segment...
Jul 28 00:45:02.293125122 (26h53m18.074058491s) [seg_pos={0 0}, upload=25659]: Done begin upload segment.
Jul 28 00:45:02.293171689 (26h53m18.074105073s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293724212 (26h53m18.074657618s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293744577 (26h53m18.074677964s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294292781 (26h53m18.075226212s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294343706 (26h53m18.075277094s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294756878 (26h53m18.075690269s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294891918 (26h53m18.075825306s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295221121 (26h53m18.076154513s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295558583 (26h53m18.076491975s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295604310 (26h53m18.0765377s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295991730 (26h53m18.07692512s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.296088221 (26h53m18.077021615s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.301695563 (26h53m18.082628952s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302203775 (26h53m18.083137164s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302629001 (26h53m18.083562389s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302995157 (26h53m18.083928524s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.418208178 (26h53m18.19914155s) [upload=25659]: Next returned nil segment
Jul 28 00:45:02.418226832 (26h53m18.19916021s) [upload=25659]: Waiting for error group managing segments...
Jul 28 00:45:02.424117363 (26h53m18.205050731s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.427234519 (26h53m18.208167896s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.430114806 (26h53m18.211048185s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.435624120 (26h53m18.216557505s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436119655 (26h53m18.217053027s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436405218 (26h53m18.217338582s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436887494 (26h53m18.217820863s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436998726 (26h53m18.217932114s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437527079 (26h53m18.218460459s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437625274 (26h53m18.218558642s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439469614 (26h53m18.220402995s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439557143 (26h53m18.220490506s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:32.221331076 (26h53m48.002264449s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: write tcp 159.223.143.39:33378->104.36.82.16:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221411781 (26h53m48.002345162s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221457184 (26h53m48.002390547s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: rpc: dial tcp 104.36.82.13:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221473904 (26h53m48.002407294s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:49546->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221516309 (26h53m48.002449671s) [seg_pos={0 0}, upload=25659]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul 28 00:45:32.251981717 (26h53m48.032915086s) [upload=25659]: Done uploading segments...
Jul 28 00:45:32.252005327 (26h53m48.032938704s) [upload=25659]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 371.260594ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip: 
8 out of 60 is failed (threshold=3) [.X...XX.......X...........X...................X....XX.......]
duration: 
403.546538ms
FAIL (2025-07-28 00:58:09 UTC) 
orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 403.581976ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)
antiblip: 
51 out of 60 is failed (threshold=3) [........XXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration: 
1m0.032577832s
FAIL (2025-07-28 00:57:32 UTC) 
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 14 but got 0)
======LOGS======
41]: Uploading piece...
Jul 28 00:56:32.841154135 (27h4m48.622087515s) [noise=true , address=104.36.82.44:4885    , piece_id=EN2R6POSTZH7PYUTSEEMUWDOBJT3OHTZJOYHUQEP5BB3KPMDRRHQ, seg_pos={0 0}, upload=25841]: Uploading piece...
Jul 28 00:56:32.841545344 (27h4m48.622478725s) [noise=true , address=104.36.82.22:4543    , piece_id=OMTBRKLQQKAOEA3234JJUJAPLTFMU64LUATYRFSAQA4EZON5JOKQ, seg_pos={0 0}, upload=25841]: Uploading piece...
Jul 28 00:56:32.841919622 (27h4m48.622853007s) [noise=true , address=104.36.82.42:4817    , piece_id=JUMDUD6P7RILIFOE26J22BWXH2RJOPOIKLXB5HLGL5RDSGYN2ZGQ, seg_pos={0 0}, upload=25841]: Uploading piece...
Jul 28 00:56:32.842094106 (27h4m48.62302749s) [noise=true , address=104.36.82.47:4533    , piece_id=QF5N4P3HRCNM3AQ76NQ6WSCDC36X52TQ4KZPCLGYOP42GXALF6WA, seg_pos={0 0}, upload=25841]: Uploading piece...
Jul 28 00:56:32.842260702 (27h4m48.623194083s) [noise=true , address=104.36.82.41:4787    , piece_id=LKJ6LK4LDYN42TQO5C7XGTSS6NN5OMBMHJR776FRFNCOZZA5DWFQ, seg_pos={0 0}, upload=25841]: Uploading piece...
Jul 28 00:56:32.842578244 (27h4m48.623511622s) [noise=true , address=104.36.82.9:4016     , piece_id=S74G2IXP3KRQ735JMVOITVRXMJON7U7JJVTUPCV7WWQZAXCBHKOQ, seg_pos={0 0}, upload=25841]: Uploading piece...
Jul 28 00:56:32.842892673 (27h4m48.623826057s) [noise=true , address=104.36.82.12:4179    , piece_id=C5G5PD6AKYOCBYULKNLSCDYRIYWWBHOCVC53HB5TPNI4WZS5DFMA, seg_pos={0 0}, upload=25841]: Uploading piece...
Jul 28 00:56:32.843265828 (27h4m48.624199214s) [noise=true , address=104.36.82.14:4237    , piece_id=7IKXJKGLOXKARCX5SUAR7HH4YICY43TURWHC2HJ2RVBLF4KUFEAQ, seg_pos={0 0}, upload=25841]: Uploading piece...
Jul 28 00:57:32.766033226 (27h5m48.546966604s) [upload=25841]: Next segment err: context deadline exceeded
Jul 28 00:57:32.767276530 (27h5m48.548209898s) [noise=true , address=104.36.82.55:4199    , piece_id=STFQPRUDKJUOJBYVQ4NNLWQCT7WZZFOQQFVNWFPFTYBKA7PZKCCQ, seg_pos={0 0}, upload=25841]: Done uploading piece. err: ecclient: upload failed (node:1mCH3fNEMd64WztUYYRCE2KUdJgLxWGoPUfEe4KaF34YhUxUXx, address:104.36.82.55:4199): protocol: rpc: dial tcp 104.36.82.55:4199: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:57:32.767345262 (27h5m48.548278628s) [noise=true , address=104.36.82.44:4885    , piece_id=EN2R6POSTZH7PYUTSEEMUWDOBJT3OHTZJOYHUQEP5BB3KPMDRRHQ, seg_pos={0 0}, upload=25841]: Done uploading piece. err: ecclient: upload failed (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG, address:104.36.82.44:4885): protocol: rpc: dial tcp 104.36.82.44:4885: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:57:32.767405943 (27h5m48.548339325s) [noise=true , address=104.36.82.22:4543    , piece_id=OMTBRKLQQKAOEA3234JJUJAPLTFMU64LUATYRFSAQA4EZON5JOKQ, seg_pos={0 0}, upload=25841]: Done uploading piece. err: ecclient: upload failed (node:19iUYyo8cGDLGDnvdNo2Qfj1K6tp6EJWF5U6rXD4gJ7Hv7Gsch, address:104.36.82.22:4543): protocol: rpc: dial tcp 104.36.82.22:4543: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:57:32.767479041 (27h5m48.548412417s) [noise=true , address=104.36.82.56:4252    , piece_id=XS47J2KP4VMZXA63BNOZ5JBN7EVNQZRMTWNZVNJFCCCQWWTX353A, seg_pos={0 0}, upload=25841]: Done uploading piece. err: ecclient: upload failed (node:1UDFNiqXb4riRuewJDPrKFykVWLiryNKqgkH6uCZszz6Sr8naB, address:104.36.82.56:4252): protocol: rpc: dial tcp 104.36.82.56:4252: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:57:32.767533085 (27h5m48.54846645s) [noise=true , address=104.36.82.10:4101    , piece_id=7IVLZL5FVAQUWZFCU7IN2H7IQIR674X2TODR42BQAG7XERXHZGUA, seg_pos={0 0}, upload=25841]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: rpc: dial tcp 104.36.82.10:4101: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:57:32.767586576 (27h5m48.548519941s) [noise=true , address=104.36.82.52:4113    , piece_id=4JLBJM2F6PAY245PCTXQ55PDBVAXUNAGHDGJX63735SYW7FSFDUA, seg_pos={0 0}, upload=25841]: Done uploading piece. err: ecclient: upload failed (node:1brZ4RtcK5J3xCPmnbsTccF9hrnj8tqhV1WyMqqrjuitgSGMou, address:104.36.82.52:4113): protocol: rpc: dial tcp 104.36.82.52:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:57:32.767649316 (27h5m48.548582696s) [noise=true , address=104.36.82.9:4016     , piece_id=S74G2IXP3KRQ735JMVOITVRXMJON7U7JJVTUPCV7WWQZAXCBHKOQ, seg_pos={0 0}, upload=25841]: Done uploading piece. err: ecclient: upload failed (node:12vV7puUjzyuxvoUYvc8dASE6bQwPTFeZSW4ZakTbBYWDfGtQoV, address:104.36.82.9:4016): protocol: rpc: dial tcp 104.36.82.9:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:57:32.767709278 (27h5m48.548642643s) [noise=true , address=104.36.82.14:4237    , piece_id=7IKXJKGLOXKARCX5SUAR7HH4YICY43TURWHC2HJ2RVBLF4KUFEAQ, seg_pos={0 0}, upload=25841]: Done uploading piece. err: ecclient: upload failed (node:1o2jLTwKNSjxjFEmckxkG1c5EuySK3o3sawGEikd2Ttz4CWfLa, address:104.36.82.14:4237): protocol: write tcp 159.223.143.39:42824->104.36.82.14:4237: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:57:32.767754499 (27h5m48.54868786s) [noise=true , address=104.36.82.51:4054    , piece_id=HUR33WBPJNEGKAJ7DOMNQ7AR4CWYA5FZGEDC2MB3MXBZVD33IFDA, seg_pos={0 0}, upload=25841]: Done uploading piece. err: ecclient: upload failed (node:1opDYSgBqBFTN6wZ5mrN5pJsnt9AH4y5sDr1yxE3Cjxg3vbq3v, address:104.36.82.51:4054): protocol: write tcp 159.223.143.39:60430->104.36.82.51:4054: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:57:32.767972690 (27h5m48.548906077s) [noise=true , address=104.36.82.54:4173    , piece_id=MHE65XCIYN6QMXE7O54MC4JIXBNRO5OE5WEPTENUUC2RXM2RNNXA, seg_pos={0 0}, upload=25841]: Done uploading piece. err: ecclient: upload failed (node:1w2v4iyYtJtcdz7shYhG8CjCDQ1dWjeJVwY5Ubi7Ex5Y38e2i1, address:104.36.82.54:4173): protocol: rpc: dial tcp 104.36.82.54:4173: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:57:32.768019426 (27h5m48.548952802s) [noise=true , address=104.36.82.53:4131    , piece_id=HJZXZNWHR4UIYPDBOMM6CD6YRBHE34FH2L3KDRXPFDFM7BQDZLWA, seg_pos={0 0}, upload=25841]: Done uploading piece. err: ecclient: upload failed (node:1uhyc5ozrF9ndeEnQXtjc9o3tsYiAnW1dzeuHV3kv1JS4T4a8C, address:104.36.82.53:4131): protocol: write tcp 159.223.143.39:47086->104.36.82.53:4131: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:57:32.768085696 (27h5m48.549019087s) [noise=true , address=104.36.82.20:4450    , piece_id=PFUTGLHUUSMYO6S7KYDAPF4MULOZDWELZQMRK3YN24NYJO6I6NPQ, seg_pos={0 0}, upload=25841]: Done uploading piece. err: ecclient: upload failed (node:12UeL1LySKQySa8wjvG4FBs69cE1iWkW4Diz83nu1PxBCKxyvki, address:104.36.82.20:4450): protocol: rpc: dial tcp 104.36.82.20:4450: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:57:32.768173183 (27h5m48.549106563s) [noise=true , address=104.36.82.13:4201    , piece_id=NGPC4FLQBA7GFYA4XKEFBNID6VJQGX3KLA4H63RU3HOHBQEPR3QQ, seg_pos={0 0}, upload=25841]: Done uploading piece. err: ecclient: upload failed (node:12VTXMHwWTyB6H6RRCAk7uupYBGhhDcyngUBx7cgUjjYA1vegL6, address:104.36.82.13:4201): protocol: write tcp 159.223.143.39:40162->104.36.82.13:4201: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:57:32.768287431 (27h5m48.549220837s) [noise=true , address=104.36.82.42:4817    , piece_id=JUMDUD6P7RILIFOE26J22BWXH2RJOPOIKLXB5HLGL5RDSGYN2ZGQ, seg_pos={0 0}, upload=25841]: Done uploading piece. err: ecclient: upload failed (node:1w1W2uV2kFMfxwSwB5o1rZbd4VGVR2XA7sk3NLWRQcZoBnheE7, address:104.36.82.42:4817): protocol: rpc: dial tcp 104.36.82.42:4817: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:57:32.768325994 (27h5m48.549259378s) [noise=true , address=104.36.82.41:4787    , piece_id=LKJ6LK4LDYN42TQO5C7XGTSS6NN5OMBMHJR776FRFNCOZZA5DWFQ, seg_pos={0 0}, upload=25841]: Done uploading piece. err: ecclient: upload failed (node:12aKqg1aNsv8Yi7fE4asGz8bijcPewWDvUukKcf5SFnzgARju43, address:104.36.82.41:4787): protocol: rpc: dial tcp 104.36.82.41:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:57:32.768377498 (27h5m48.549310891s) [noise=true , address=104.36.82.12:4179    , piece_id=C5G5PD6AKYOCBYULKNLSCDYRIYWWBHOCVC53HB5TPNI4WZS5DFMA, seg_pos={0 0}, upload=25841]: Done uploading piece. err: ecclient: upload failed (node:129pP9f3rFfTvP6MhG894N7WHuJRZP8aWhzJf1zBwZ76wbzVbZa, address:104.36.82.12:4179): protocol: rpc: dial tcp 104.36.82.12:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:57:32.768407208 (27h5m48.549340593s) [noise=true , address=104.36.82.47:4533    , piece_id=QF5N4P3HRCNM3AQ76NQ6WSCDC36X52TQ4KZPCLGYOP42GXALF6WA, seg_pos={0 0}, upload=25841]: Done uploading piece. err: ecclient: upload failed (node:1JUUdAnaUPAE8tkzPvVCokFbZMwmhqZvwKHXReK9VU2QeDqavR, address:104.36.82.47:4533): protocol: rpc: dial tcp 104.36.82.47:4533: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:57:32.768451715 (27h5m48.549385093s) [noise=true , address=104.36.82.11:4136    , piece_id=XBLRJLOIAWGMAZQUSYUZCIPYZQIMV3CC77HXZHFXOVVQ7UZBMYPQ, seg_pos={0 0}, upload=25841]: Done uploading piece. err: ecclient: upload failed (node:127n8CMbZQ7haiv6tGHrVg9V8Yf12sYkcbL9mEbSpQ4WkT9isnp, address:104.36.82.11:4136): protocol: rpc: dial tcp 104.36.82.11:4136: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:57:32.768487989 (27h5m48.549421371s) [seg_pos={0 0}, upload=25841]: Done waiting for segment. successful: 0 optimal: 14 errs: <nil>
Duration: 1m0.032609712s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
6 out of 60 is failed (threshold=3) [.......X.............XX...........X...X...X.................]
duration: 
496.710041ms
FAIL (2025-07-28 00:57:03 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
0}, upload=25651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 159.223.143.39:39890->104.36.82.7:4909: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:44:18.975782256 (26h52m34.756715638s) [seg_pos={0 0}, upload=25651]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 28 00:44:18.975871874 (26h52m34.756805277s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 28 00:44:18.975878645 (26h52m34.756812034s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 28 00:44:19.019856339 (26h52m34.800789708s) [upload=25651]: Done uploading segments...
Jul 28 00:44:19.019879905 (26h52m34.800813283s) [upload=25651]: Upload finished. err: <nil>
Jul 28 00:45:02.257192553 (26h53m18.03812594s) [upload=25659]: Starting upload
Jul 28 00:45:02.257306836 (26h53m18.038240216s) [upload=25659]: Done starting upload
Jul 28 00:45:02.257434360 (26h53m18.038367739s) [upload=25659]: Uploading segments...
Jul 28 00:45:02.257858888 (26h53m18.038792251s) [upload=25659]: Got next segment. Inline: false
Jul 28 00:45:02.257951856 (26h53m18.038885235s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 28 00:45:02.257959915 (26h53m18.038893293s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 28 00:45:02.293048419 (26h53m18.073981806s) [seg_pos={0 0}, upload=25659]: Begin upload segment...
Jul 28 00:45:02.293125122 (26h53m18.074058491s) [seg_pos={0 0}, upload=25659]: Done begin upload segment.
Jul 28 00:45:02.293171689 (26h53m18.074105073s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293724212 (26h53m18.074657618s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293744577 (26h53m18.074677964s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294292781 (26h53m18.075226212s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294343706 (26h53m18.075277094s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294756878 (26h53m18.075690269s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294891918 (26h53m18.075825306s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295221121 (26h53m18.076154513s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295558583 (26h53m18.076491975s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295604310 (26h53m18.0765377s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295991730 (26h53m18.07692512s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.296088221 (26h53m18.077021615s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.301695563 (26h53m18.082628952s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302203775 (26h53m18.083137164s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302629001 (26h53m18.083562389s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302995157 (26h53m18.083928524s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.418208178 (26h53m18.19914155s) [upload=25659]: Next returned nil segment
Jul 28 00:45:02.418226832 (26h53m18.19916021s) [upload=25659]: Waiting for error group managing segments...
Jul 28 00:45:02.424117363 (26h53m18.205050731s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.427234519 (26h53m18.208167896s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.430114806 (26h53m18.211048185s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.435624120 (26h53m18.216557505s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436119655 (26h53m18.217053027s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436405218 (26h53m18.217338582s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436887494 (26h53m18.217820863s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436998726 (26h53m18.217932114s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437527079 (26h53m18.218460459s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437625274 (26h53m18.218558642s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439469614 (26h53m18.220402995s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439557143 (26h53m18.220490506s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:32.221331076 (26h53m48.002264449s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: write tcp 159.223.143.39:33378->104.36.82.16:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221411781 (26h53m48.002345162s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221457184 (26h53m48.002390547s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: rpc: dial tcp 104.36.82.13:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221473904 (26h53m48.002407294s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:49546->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221516309 (26h53m48.002449671s) [seg_pos={0 0}, upload=25659]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul 28 00:45:32.251981717 (26h53m48.032915086s) [upload=25659]: Done uploading segments...
Jul 28 00:45:32.252005327 (26h53m48.032938704s) [upload=25659]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 496.748451ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip: 
8 out of 60 is failed (threshold=3) [..X...XX.......X...........X...................X....XX......]
duration: 
441.136915ms
FAIL (2025-07-28 00:56:57 UTC) 
orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 441.166309ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)
antiblip: 
50 out of 60 is failed (threshold=3) [.........XXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration: 
1m0.035414757s
FAIL (2025-07-28 00:56:32 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 2)
======LOGS======
 (27h3m48.496632389s) [noise=true , address=104.36.82.47:4551    , piece_id=RSAZ37Y5YDVA5WAH3RFRVQNCZQOOQXAM4QYKOPZQK5HLCPEULMDA, seg_pos={0 0}, upload=25824]: Uploading piece...
Jul 28 00:55:32.716002815 (27h3m48.496936191s) [noise=true , address=104.36.82.5:4844     , piece_id=7GLAU64CZBHVXXS7R7COGKSXULOXXNKWPZVOM7GLN5QDCCO5GGXA, seg_pos={0 0}, upload=25824]: Uploading piece...
Jul 28 00:55:32.716358068 (27h3m48.497291453s) [noise=true , address=104.36.82.44:4883    , piece_id=ERFU36YGIGIIOIBUOS3P5TAVMVNFF6NZTGJGOEHI3VFBXV76QPIQ, seg_pos={0 0}, upload=25824]: Uploading piece...
Jul 28 00:55:32.716755937 (27h3m48.497689304s) [noise=true , address=104.36.82.20:4455    , piece_id=ZXJQ3L456VUTL6BDBJX7CCF6SFCL7ZFQLOU7KQTX4GMWNW7E5SEQ, seg_pos={0 0}, upload=25824]: Uploading piece...
Jul 28 00:55:32.717088303 (27h3m48.498021684s) [noise=true , address=104.36.82.15:4267    , piece_id=LRBK3DMBPDDTUQJMIYUX4IW3FUUWDOBAILYX4ZZDBFDKQH34UV2Q, seg_pos={0 0}, upload=25824]: Uploading piece...
Jul 28 00:55:32.717405599 (27h3m48.498338981s) [noise=true , address=79.127.201.215:4359  , piece_id=E74DAGGPD7GOEUKV5DZM4V6RIRMI6BL56GPYZUO2RVONMOMG5JMA, seg_pos={0 0}, upload=25824]: Uploading piece...
Jul 28 00:55:32.717730995 (27h3m48.498664372s) [noise=true , address=104.36.82.14:4228    , piece_id=3DUTGZ7WAUFYUMWBPPSH3FNRJBS3BXHANTZBFEW3YT4XJFBKZAFQ, seg_pos={0 0}, upload=25824]: Uploading piece...
Jul 28 00:55:32.718029531 (27h3m48.498962907s) [noise=true , address=104.36.82.10:4086    , piece_id=NJIIAOY5M65EGJBT3RUKXTWLHDAI4HGX5Z7AKRCMCSMRHZE3NEGA, seg_pos={0 0}, upload=25824]: Uploading piece...
Jul 28 00:55:32.718302916 (27h3m48.499236294s) [noise=true , address=104.36.82.11:4132    , piece_id=U743ADKVJ3TXPCSM4K7INPOYJKZOYCLRTNNNOBDXN5K7UNSE5FCA, seg_pos={0 0}, upload=25824]: Uploading piece...
Jul 28 00:55:32.914628881 (27h3m48.695562263s) [upload=25824]: Next returned nil segment
Jul 28 00:55:32.914652605 (27h3m48.695585991s) [upload=25824]: Waiting for error group managing segments...
Jul 28 00:55:33.644615097 (27h3m49.425548479s) [noise=true , address=64.49.13.27:4957     , piece_id=Q4GYU55OYTQ4BQIUM77M7SPLJCOVPWVY5ITB6I6WBBDEMEZU4P4Q, seg_pos={0 0}, upload=25824]: Done uploading piece. err: <nil>
Jul 28 00:55:33.711806109 (27h3m49.492739477s) [noise=true , address=79.127.201.215:4359  , piece_id=E74DAGGPD7GOEUKV5DZM4V6RIRMI6BL56GPYZUO2RVONMOMG5JMA, seg_pos={0 0}, upload=25824]: Done uploading piece. err: <nil>
Jul 28 00:56:32.648173278 (27h4m48.429106656s) [noise=true , address=104.36.82.44:4883    , piece_id=ERFU36YGIGIIOIBUOS3P5TAVMVNFF6NZTGJGOEHI3VFBXV76QPIQ, seg_pos={0 0}, upload=25824]: Done uploading piece. err: ecclient: upload failed (node:12cx1avL4umDm2pzJS1rNZqeDZjZMWnMtNsyNTdMdkxhpa1ByUN, address:104.36.82.44:4883): protocol: rpc: dial tcp 104.36.82.44:4883: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:56:32.648427919 (27h4m48.429361295s) [noise=true , address=104.36.82.14:4228    , piece_id=3DUTGZ7WAUFYUMWBPPSH3FNRJBS3BXHANTZBFEW3YT4XJFBKZAFQ, seg_pos={0 0}, upload=25824]: Done uploading piece. err: ecclient: upload failed (node:12pkADNFaYa2SqFALQdxwf1CfR1ktUwshQkw7iBJzhdhaRYaszF, address:104.36.82.14:4228): protocol: rpc: dial tcp 104.36.82.14:4228: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:56:32.648437845 (27h4m48.429371228s) [noise=true , address=104.36.82.20:4455    , piece_id=ZXJQ3L456VUTL6BDBJX7CCF6SFCL7ZFQLOU7KQTX4GMWNW7E5SEQ, seg_pos={0 0}, upload=25824]: Done uploading piece. err: ecclient: upload failed (node:1X8uFZXHVMycvk6tELmYQyWoAS2sSFmNNWhvQFApvoWXuXFcj4, address:104.36.82.20:4455): protocol: rpc: dial tcp 104.36.82.20:4455: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:56:32.648492904 (27h4m48.42942628s) [noise=true , address=104.36.82.10:4086    , piece_id=NJIIAOY5M65EGJBT3RUKXTWLHDAI4HGX5Z7AKRCMCSMRHZE3NEGA, seg_pos={0 0}, upload=25824]: Done uploading piece. err: ecclient: upload failed (node:1fMyQ9ojAvrZhG4sLwG3wzNJZihg8ZwWKBJmgWpK3Ckx5YXrg3, address:104.36.82.10:4086): protocol: write tcp 159.223.143.39:38564->104.36.82.10:4086: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:56:32.648501424 (27h4m48.429434804s) [noise=true , address=104.36.82.15:4267    , piece_id=LRBK3DMBPDDTUQJMIYUX4IW3FUUWDOBAILYX4ZZDBFDKQH34UV2Q, seg_pos={0 0}, upload=25824]: Done uploading piece. err: ecclient: upload failed (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7, address:104.36.82.15:4267): protocol: rpc: dial tcp 104.36.82.15:4267: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:56:32.648549899 (27h4m48.429483275s) [noise=true , address=104.36.82.53:4135    , piece_id=LEZ3RWKERDB7TVMNB7OGG477MFJB5CTIEPH7HEU3U4QLYXP2YSWQ, seg_pos={0 0}, upload=25824]: Done uploading piece. err: ecclient: upload failed (node:128j2QoFq4UDWCXjV8uChmN9KB1sQMnhdF6hGgNQygdqHvw9tPj, address:104.36.82.53:4135): protocol: rpc: dial tcp 104.36.82.53:4135: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:56:32.648562693 (27h4m48.429496069s) [noise=true , address=104.36.82.11:4132    , piece_id=U743ADKVJ3TXPCSM4K7INPOYJKZOYCLRTNNNOBDXN5K7UNSE5FCA, seg_pos={0 0}, upload=25824]: Done uploading piece. err: ecclient: upload failed (node:17856PRHooRiPyrBYLJggh5uYpdWA9CnGxgCrmVesToNu7FeLb, address:104.36.82.11:4132): protocol: rpc: dial tcp 104.36.82.11:4132: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:56:32.648611712 (27h4m48.429545088s) [noise=true , address=104.36.82.40:4953    , piece_id=QJQW4VRK5CUVVD5EAWLYIJSNCDV5VIDUTTKBVFY44W6Z5OVB4QEQ, seg_pos={0 0}, upload=25824]: Done uploading piece. err: ecclient: upload failed (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn, address:104.36.82.40:4953): protocol: rpc: dial tcp 104.36.82.40:4953: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:56:32.648623129 (27h4m48.429556509s) [noise=true , address=104.36.82.54:4165    , piece_id=5JMNWFC4FMJL2CXRQMWPVVYTWH3DT5K6VTIRBOSAWLEA7YT5YDHA, seg_pos={0 0}, upload=25824]: Done uploading piece. err: ecclient: upload failed (node:12wGUMXVEubEtki4qGnjGEsbYFB1iUauZcu3oDRKrCQsJMV3bV2, address:104.36.82.54:4165): protocol: rpc: dial tcp 104.36.82.54:4165: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:56:32.648670150 (27h4m48.429603528s) [noise=true , address=104.36.82.5:4844     , piece_id=7GLAU64CZBHVXXS7R7COGKSXULOXXNKWPZVOM7GLN5QDCCO5GGXA, seg_pos={0 0}, upload=25824]: Done uploading piece. err: ecclient: upload failed (node:12iovpNdbidkiv9mEHaAPjBubGmoqr4sUJWm8Cb64LzS6wuwLWa, address:104.36.82.5:4844): protocol: rpc: dial tcp 104.36.82.5:4844: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:56:32.648680553 (27h4m48.429613933s) [noise=true , address=104.36.82.55:4206    , piece_id=SV6XWXI4EELBHT2FD6TSYJFQICWAUAV3EJRHK7NNSFPOHIJSEOPA, seg_pos={0 0}, upload=25824]: Done uploading piece. err: ecclient: upload failed (node:12Mp4ZzYgrP3ZSXDeHbSTXaAJMPgGKUqVjdNtyfeKE5qBMAzBke, address:104.36.82.55:4206): protocol: rpc: dial tcp 104.36.82.55:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:56:32.648727225 (27h4m48.429660603s) [noise=true , address=104.36.82.46:4494    , piece_id=37DROWO4OVFKTKIJF26SGUN6SANMYMVWUWAW2ZWQOBD7OEOH3I3A, seg_pos={0 0}, upload=25824]: Done uploading piece. err: ecclient: upload failed (node:17b8LdpavJCCFXEBj7ndUKR6sge6Ntv96cGK5TmahF3MtwTV9z, address:104.36.82.46:4494): protocol: rpc: dial tcp 104.36.82.46:4494: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:56:32.648761284 (27h4m48.429694662s) [noise=true , address=104.36.82.3:4781     , piece_id=XCY2VJWLO2DPA3L3CUX6VT4P6OWCCI5RXTOXX6A6RDSNN47BFWUA, seg_pos={0 0}, upload=25824]: Done uploading piece. err: ecclient: upload failed (node:1Y6iagayTdoSQtXrAdeBrvrck8qMNRAmLogi5jUG8bcLrBEswn, address:104.36.82.3:4781): protocol: rpc: dial tcp 104.36.82.3:4781: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:56:32.648809613 (27h4m48.429742978s) [noise=true , address=104.36.82.15:4278    , piece_id=HGWBGDZEDJ2GSK7LII2KISHXDF5JNGYKT5LPSZTRM5QMOLNFRQGA, seg_pos={0 0}, upload=25824]: Done uploading piece. err: ecclient: upload failed (node:12qpbyKnJ8qXmos7gdxd46F8QXx98LUwtKD6mryaXPBDD5WFSwu, address:104.36.82.15:4278): protocol: write tcp 159.223.143.39:48534->104.36.82.15:4278: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:56:32.648815977 (27h4m48.429749355s) [noise=true , address=104.36.82.16:4321    , piece_id=WNIMAHQLNERZ6G2DFF6E73TUVKZXQHYC2R3YBI5EVTFEHI64FNKA, seg_pos={0 0}, upload=25824]: Done uploading piece. err: ecclient: upload failed (node:12edsHqD9iRBDgMU8vYhSM1EWG8Sn57GZ8TPiTytri3rp7W9Eae, address:104.36.82.16:4321): protocol: rpc: dial tcp 104.36.82.16:4321: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:56:32.648858863 (27h4m48.429792241s) [noise=true , address=104.36.82.47:4551    , piece_id=RSAZ37Y5YDVA5WAH3RFRVQNCZQOOQXAM4QYKOPZQK5HLCPEULMDA, seg_pos={0 0}, upload=25824]: Done uploading piece. err: ecclient: upload failed (node:1UhJvSMtNqJDak4TM7eUgz1N7yCn4ChmRRTe7nXdffTNsS1pch, address:104.36.82.47:4551): protocol: write tcp 159.223.143.39:38168->104.36.82.47:4551: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:56:32.648891689 (27h4m48.429825074s) [seg_pos={0 0}, upload=25824]: Done waiting for segment. successful: 2 optimal: 14 errs: <nil>
Jul 28 00:56:32.678942853 (27h4m48.459876217s) [upload=25824]: Done uploading segments...
Jul 28 00:56:32.679094906 (27h4m48.460028284s) [upload=25824]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 2)
Duration: 1m0.03543321s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip: 
8 out of 60 is failed (threshold=3) [...X...XX.......X...........X...................X....XX.....]
duration: 
384.586459ms
FAIL (2025-07-28 00:56:23 UTC) 
orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 384.681079ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
6 out of 60 is failed (threshold=3) [........X.............XX...........X...X...X................]
duration: 
397.881224ms
FAIL (2025-07-28 00:56:15 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
0}, upload=25651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 159.223.143.39:39890->104.36.82.7:4909: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:44:18.975782256 (26h52m34.756715638s) [seg_pos={0 0}, upload=25651]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 28 00:44:18.975871874 (26h52m34.756805277s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 28 00:44:18.975878645 (26h52m34.756812034s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 28 00:44:19.019856339 (26h52m34.800789708s) [upload=25651]: Done uploading segments...
Jul 28 00:44:19.019879905 (26h52m34.800813283s) [upload=25651]: Upload finished. err: <nil>
Jul 28 00:45:02.257192553 (26h53m18.03812594s) [upload=25659]: Starting upload
Jul 28 00:45:02.257306836 (26h53m18.038240216s) [upload=25659]: Done starting upload
Jul 28 00:45:02.257434360 (26h53m18.038367739s) [upload=25659]: Uploading segments...
Jul 28 00:45:02.257858888 (26h53m18.038792251s) [upload=25659]: Got next segment. Inline: false
Jul 28 00:45:02.257951856 (26h53m18.038885235s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 28 00:45:02.257959915 (26h53m18.038893293s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 28 00:45:02.293048419 (26h53m18.073981806s) [seg_pos={0 0}, upload=25659]: Begin upload segment...
Jul 28 00:45:02.293125122 (26h53m18.074058491s) [seg_pos={0 0}, upload=25659]: Done begin upload segment.
Jul 28 00:45:02.293171689 (26h53m18.074105073s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293724212 (26h53m18.074657618s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293744577 (26h53m18.074677964s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294292781 (26h53m18.075226212s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294343706 (26h53m18.075277094s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294756878 (26h53m18.075690269s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294891918 (26h53m18.075825306s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295221121 (26h53m18.076154513s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295558583 (26h53m18.076491975s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295604310 (26h53m18.0765377s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295991730 (26h53m18.07692512s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.296088221 (26h53m18.077021615s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.301695563 (26h53m18.082628952s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302203775 (26h53m18.083137164s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302629001 (26h53m18.083562389s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302995157 (26h53m18.083928524s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.418208178 (26h53m18.19914155s) [upload=25659]: Next returned nil segment
Jul 28 00:45:02.418226832 (26h53m18.19916021s) [upload=25659]: Waiting for error group managing segments...
Jul 28 00:45:02.424117363 (26h53m18.205050731s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.427234519 (26h53m18.208167896s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.430114806 (26h53m18.211048185s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.435624120 (26h53m18.216557505s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436119655 (26h53m18.217053027s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436405218 (26h53m18.217338582s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436887494 (26h53m18.217820863s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436998726 (26h53m18.217932114s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437527079 (26h53m18.218460459s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437625274 (26h53m18.218558642s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439469614 (26h53m18.220402995s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439557143 (26h53m18.220490506s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:32.221331076 (26h53m48.002264449s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: write tcp 159.223.143.39:33378->104.36.82.16:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221411781 (26h53m48.002345162s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221457184 (26h53m48.002390547s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: rpc: dial tcp 104.36.82.13:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221473904 (26h53m48.002407294s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:49546->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221516309 (26h53m48.002449671s) [seg_pos={0 0}, upload=25659]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul 28 00:45:32.251981717 (26h53m48.032915086s) [upload=25659]: Done uploading segments...
Jul 28 00:45:32.252005327 (26h53m48.032938704s) [upload=25659]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 397.915284ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
7 out of 60 is failed (threshold=3) [X........X.............XX...........X...X...X...............]
duration: 
396.857868ms
FAIL (2025-07-28 00:55:55 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
0}, upload=25651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 159.223.143.39:39890->104.36.82.7:4909: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:44:18.975782256 (26h52m34.756715638s) [seg_pos={0 0}, upload=25651]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 28 00:44:18.975871874 (26h52m34.756805277s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 28 00:44:18.975878645 (26h52m34.756812034s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 28 00:44:19.019856339 (26h52m34.800789708s) [upload=25651]: Done uploading segments...
Jul 28 00:44:19.019879905 (26h52m34.800813283s) [upload=25651]: Upload finished. err: <nil>
Jul 28 00:45:02.257192553 (26h53m18.03812594s) [upload=25659]: Starting upload
Jul 28 00:45:02.257306836 (26h53m18.038240216s) [upload=25659]: Done starting upload
Jul 28 00:45:02.257434360 (26h53m18.038367739s) [upload=25659]: Uploading segments...
Jul 28 00:45:02.257858888 (26h53m18.038792251s) [upload=25659]: Got next segment. Inline: false
Jul 28 00:45:02.257951856 (26h53m18.038885235s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 28 00:45:02.257959915 (26h53m18.038893293s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 28 00:45:02.293048419 (26h53m18.073981806s) [seg_pos={0 0}, upload=25659]: Begin upload segment...
Jul 28 00:45:02.293125122 (26h53m18.074058491s) [seg_pos={0 0}, upload=25659]: Done begin upload segment.
Jul 28 00:45:02.293171689 (26h53m18.074105073s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293724212 (26h53m18.074657618s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293744577 (26h53m18.074677964s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294292781 (26h53m18.075226212s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294343706 (26h53m18.075277094s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294756878 (26h53m18.075690269s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294891918 (26h53m18.075825306s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295221121 (26h53m18.076154513s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295558583 (26h53m18.076491975s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295604310 (26h53m18.0765377s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295991730 (26h53m18.07692512s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.296088221 (26h53m18.077021615s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.301695563 (26h53m18.082628952s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302203775 (26h53m18.083137164s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302629001 (26h53m18.083562389s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302995157 (26h53m18.083928524s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.418208178 (26h53m18.19914155s) [upload=25659]: Next returned nil segment
Jul 28 00:45:02.418226832 (26h53m18.19916021s) [upload=25659]: Waiting for error group managing segments...
Jul 28 00:45:02.424117363 (26h53m18.205050731s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.427234519 (26h53m18.208167896s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.430114806 (26h53m18.211048185s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.435624120 (26h53m18.216557505s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436119655 (26h53m18.217053027s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436405218 (26h53m18.217338582s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436887494 (26h53m18.217820863s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436998726 (26h53m18.217932114s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437527079 (26h53m18.218460459s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437625274 (26h53m18.218558642s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439469614 (26h53m18.220402995s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439557143 (26h53m18.220490506s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:32.221331076 (26h53m48.002264449s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: write tcp 159.223.143.39:33378->104.36.82.16:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221411781 (26h53m48.002345162s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221457184 (26h53m48.002390547s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: rpc: dial tcp 104.36.82.13:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221473904 (26h53m48.002407294s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:49546->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221516309 (26h53m48.002449671s) [seg_pos={0 0}, upload=25659]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul 28 00:45:32.251981717 (26h53m48.032915086s) [upload=25659]: Done uploading segments...
Jul 28 00:45:32.252005327 (26h53m48.032938704s) [upload=25659]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 396.891153ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)
antiblip: 
49 out of 60 is failed (threshold=3) [..........XXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration: 
1m0.034713973s
FAIL (2025-07-28 00:55:32 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 2)
======LOGS======
399681849s) [noise=true , address=104.36.82.11:4120    , piece_id=62HSWY44QWVW4F3D4NVRV5ZQPOWV7G56OJQ5KC3S76KGMCCSCB4A, seg_pos={0 0}, upload=25807]: Uploading piece...
Jul 28 00:54:32.620750751 (27h2m48.401684136s) [noise=true , address=104.36.82.16:4327    , piece_id=DCHJFW62X3AM33W7NL5ZDH2VWDZ2CVAZ3CUW5K4ORXGVTP5KA6XA, seg_pos={0 0}, upload=25807]: Uploading piece...
Jul 28 00:54:32.621183013 (27h2m48.402116398s) [noise=true , address=104.36.82.56:4239    , piece_id=HIGBGEHE3DTUKZR6UC4TC6CHFPVZ4FTTV5MNEBSULNTA7DNXOUZA, seg_pos={0 0}, upload=25807]: Uploading piece...
Jul 28 00:54:32.621516305 (27h2m48.40244968s) [noise=true , address=79.127.201.215:4353  , piece_id=UI5EX63KUPKQNWUYORH7WKZ6D6GG7KJU7C4L6JWOVZTWCP63SYDQ, seg_pos={0 0}, upload=25807]: Uploading piece...
Jul 28 00:54:32.621740575 (27h2m48.402673959s) [noise=true , address=104.36.82.13:4204    , piece_id=SVN72NXOLUPSZH5MVC2V2ZJTIZW6BSALENAMMCQ3GB5HPSEKXPZQ, seg_pos={0 0}, upload=25807]: Uploading piece...
Jul 28 00:54:32.621923212 (27h2m48.402856931s) [noise=true , address=104.36.82.15:4271    , piece_id=GGKEJKSIW2TLXNGSAUN6CZUPYXXH6FAAFNPP5SG3VJCFA3YHHMJQ, seg_pos={0 0}, upload=25807]: Uploading piece...
Jul 28 00:54:32.622257103 (27h2m48.40319049s) [noise=true , address=64.49.13.140:4508    , piece_id=MZAUZMQWEHDG24MCNKAV3ANU6QA6G526EEZQ63S4NUQEB4KXQ3KA, seg_pos={0 0}, upload=25807]: Uploading piece...
Jul 28 00:54:32.622340911 (27h2m48.403274296s) [noise=true , address=104.36.82.54:4158    , piece_id=OXMLUVPOZJPXHRX7JVJSZXD75FHDDCJC2C3IJ7IUC2ISMMQTJBZQ, seg_pos={0 0}, upload=25807]: Uploading piece...
Jul 28 00:54:32.622831467 (27h2m48.403764852s) [noise=true , address=104.36.82.42:4806    , piece_id=QOYEX7QFZNDN542RZVOFFJWSGZFM5PAZISACVSNEDKV7IT4LKWLQ, seg_pos={0 0}, upload=25807]: Uploading piece...
Jul 28 00:54:32.623422353 (27h2m48.40435587s) [noise=true , address=104.36.82.44:4899    , piece_id=Z7F2ULHJPSENUCSX6I5VULX3MLHOXDDRU7NA4MX6DAJBLY2GHX4Q, seg_pos={0 0}, upload=25807]: Uploading piece...
Jul 28 00:54:32.849281881 (27h2m48.63021525s) [upload=25807]: Next returned nil segment
Jul 28 00:54:32.849306554 (27h2m48.630239922s) [upload=25807]: Waiting for error group managing segments...
Jul 28 00:54:33.256627663 (27h2m49.037561042s) [noise=true , address=64.49.13.140:4508    , piece_id=MZAUZMQWEHDG24MCNKAV3ANU6QA6G526EEZQ63S4NUQEB4KXQ3KA, seg_pos={0 0}, upload=25807]: Done uploading piece. err: <nil>
Jul 28 00:54:37.418360984 (27h2m53.199294346s) [noise=true , address=79.127.201.215:4353  , piece_id=UI5EX63KUPKQNWUYORH7WKZ6D6GG7KJU7C4L6JWOVZTWCP63SYDQ, seg_pos={0 0}, upload=25807]: Done uploading piece. err: <nil>
Jul 28 00:55:32.539686217 (27h3m48.320619581s) [noise=true , address=104.36.82.52:4099    , piece_id=GQAU5IVRUUBRWZTSBJ2LSD3QO7XBKA4UKORO2NAXE3DOOYCP3XPQ, seg_pos={0 0}, upload=25807]: Done uploading piece. err: ecclient: upload failed (node:122dGFWNmsuheRmfEfNGWHftPpCcWv7HHXKymgJxsye6P7ZKhZT, address:104.36.82.52:4099): protocol: rpc: dial tcp 104.36.82.52:4099: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:55:32.539783176 (27h3m48.320716542s) [noise=true , address=104.36.82.3:4780     , piece_id=M7YEFZTW2OCNQHVPC74ZWD6ZZBVUL4OVYNP4ZDWL43UOYTTCZRMQ, seg_pos={0 0}, upload=25807]: Done uploading piece. err: ecclient: upload failed (node:1i8cJRxzLNvUYqELrufixk24oDZwDHroyG5Cfy13csTPgeajRM, address:104.36.82.3:4780): protocol: rpc: dial tcp 104.36.82.3:4780: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:55:32.539865492 (27h3m48.320798855s) [noise=true , address=104.36.82.10:4094    , piece_id=C5UO3PG76W7C2CJ33ZXKNUI36PIBFHT5FPO5CQ6IT2RKEJ5HTAJQ, seg_pos={0 0}, upload=25807]: Done uploading piece. err: ecclient: upload failed (node:124zBCfZBTW69BSaGRzUyjWUBjSJhaLMZXQYnqMPGSKL4EX6HVV, address:104.36.82.10:4094): protocol: rpc: dial tcp 104.36.82.10:4094: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:55:32.539967860 (27h3m48.320901235s) [noise=true , address=104.36.82.15:4271    , piece_id=GGKEJKSIW2TLXNGSAUN6CZUPYXXH6FAAFNPP5SG3VJCFA3YHHMJQ, seg_pos={0 0}, upload=25807]: Done uploading piece. err: ecclient: upload failed (node:1T9uQjynbTYcaixhvHWCKzatRWpFxMonHFi5idiMQtvivupKSu, address:104.36.82.15:4271): protocol: rpc: dial tcp 104.36.82.15:4271: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:55:32.540055523 (27h3m48.320988906s) [noise=true , address=104.36.82.53:4146    , piece_id=LXHLIPH7SRD663QXFO4Q5WCVYIGFBFXREGO4ECTBMASM2LWJNEDA, seg_pos={0 0}, upload=25807]: Done uploading piece. err: ecclient: upload failed (node:1ecwZFchyb5HmyKwRFfLDaVGck3xDWmgrVNYadHoTLWY5Z2Qs1, address:104.36.82.53:4146): protocol: context deadline exceeded; context deadline exceeded
Jul 28 00:55:32.540122794 (27h3m48.321056163s) [noise=true , address=104.36.82.55:4228    , piece_id=MEVULCIP53SSH4IQEGIZBFZFNZLM6KM3KC5KP6R2DXYVV4YCVKWQ, seg_pos={0 0}, upload=25807]: Done uploading piece. err: ecclient: upload failed (node:12NBsd33JtAwPdcbD8vBzZUYpjiqZpRzs8y2QrYJFE245pd8T5X, address:104.36.82.55:4228): protocol: rpc: dial tcp 104.36.82.55:4228: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:55:32.540175482 (27h3m48.321108864s) [noise=true , address=104.36.82.16:4327    , piece_id=DCHJFW62X3AM33W7NL5ZDH2VWDZ2CVAZ3CUW5K4ORXGVTP5KA6XA, seg_pos={0 0}, upload=25807]: Done uploading piece. err: ecclient: upload failed (node:1qZC1tmCvCadnm1fYZxR3kc4fonZUJPzvfTKkTFiDtccW5bo9T, address:104.36.82.16:4327): protocol: context deadline exceeded; context deadline exceeded
Jul 28 00:55:32.540473343 (27h3m48.321406705s) [noise=true , address=104.36.82.13:4204    , piece_id=SVN72NXOLUPSZH5MVC2V2ZJTIZW6BSALENAMMCQ3GB5HPSEKXPZQ, seg_pos={0 0}, upload=25807]: Done uploading piece. err: ecclient: upload failed (node:12R53RRLYuJc7r9Z1W9n6guzrr5wF4T4vuGMEzjWAPBJwSCJq1g, address:104.36.82.13:4204): protocol: rpc: dial tcp 104.36.82.13:4204: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:55:32.540534885 (27h3m48.321468252s) [noise=true , address=104.36.82.44:4899    , piece_id=Z7F2ULHJPSENUCSX6I5VULX3MLHOXDDRU7NA4MX6DAJBLY2GHX4Q, seg_pos={0 0}, upload=25807]: Done uploading piece. err: ecclient: upload failed (node:12b4pZVQRnjcQNn45VTgdsz5fpjYPMAzSXGR5KJ5BP9XHMkRnRw, address:104.36.82.44:4899): protocol: rpc: dial tcp 104.36.82.44:4899: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:55:32.540620216 (27h3m48.321553595s) [noise=true , address=104.36.82.50:4029    , piece_id=HQ77U3HYKBHTJQEIA27H3IX3ZQZTKMLZ7DOSDMDNWWMCDTKQL3GQ, seg_pos={0 0}, upload=25807]: Done uploading piece. err: ecclient: upload failed (node:12AEB1BAdLzW2M7SoNL2FfNYGs97AAziQA6pgFY5uTauqy53jsX, address:104.36.82.50:4029): protocol: rpc: dial tcp 104.36.82.50:4029: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:55:32.540635425 (27h3m48.321568804s) [noise=true , address=104.36.82.56:4239    , piece_id=HIGBGEHE3DTUKZR6UC4TC6CHFPVZ4FTTV5MNEBSULNTA7DNXOUZA, seg_pos={0 0}, upload=25807]: Done uploading piece. err: ecclient: upload failed (node:126Feu3gWgog12YavTHZkkUKK8wq6RynV8zsWd3WUAUYjvQwBnc, address:104.36.82.56:4239): protocol: rpc: dial tcp 104.36.82.56:4239: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:55:32.540676872 (27h3m48.321610254s) [noise=true , address=104.36.82.2:4757     , piece_id=GB7ZENE32MZCOCBBVXVC736YUH23EXQAHITDHUA6NPRO2QI3N2LA, seg_pos={0 0}, upload=25807]: Done uploading piece. err: ecclient: upload failed (node:1uVPbSLBfJdPZqh4yFcEMQTPUwbLhT4GAM813h2nK5qa9xCw9q, address:104.36.82.2:4757): protocol: rpc: dial tcp 104.36.82.2:4757: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:55:32.540739820 (27h3m48.321673197s) [noise=true , address=104.36.82.42:4806    , piece_id=QOYEX7QFZNDN542RZVOFFJWSGZFM5PAZISACVSNEDKV7IT4LKWLQ, seg_pos={0 0}, upload=25807]: Done uploading piece. err: ecclient: upload failed (node:1tWnxuTSo4yHAJUzvVFTmwPSNb8cBJJJ17BnKd5Z3HMYf66vSC, address:104.36.82.42:4806): protocol: rpc: dial tcp 104.36.82.42:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:55:32.540761578 (27h3m48.321694956s) [noise=true , address=104.36.82.54:4158    , piece_id=OXMLUVPOZJPXHRX7JVJSZXD75FHDDCJC2C3IJ7IUC2ISMMQTJBZQ, seg_pos={0 0}, upload=25807]: Done uploading piece. err: ecclient: upload failed (node:1vJZrSvPvwebw9V2tTbvkPEmLeLjryBUnq9TL6NwdPsHU9xcq2, address:104.36.82.54:4158): protocol: write tcp 159.223.143.39:48832->104.36.82.54:4158: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:55:32.540776348 (27h3m48.32170971s) [noise=true , address=104.36.82.11:4120    , piece_id=62HSWY44QWVW4F3D4NVRV5ZQPOWV7G56OJQ5KC3S76KGMCCSCB4A, seg_pos={0 0}, upload=25807]: Done uploading piece. err: ecclient: upload failed (node:123jcA328VCSUaWMKvSaeXip1U6Sc4SSvuRaoHF8fvsF6Tt6DHW, address:104.36.82.11:4120): protocol: rpc: dial tcp 104.36.82.11:4120: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:55:32.540803480 (27h3m48.321736847s) [noise=true , address=104.36.82.12:4175    , piece_id=6BUUJHQ63N7D7PAHL6ZPW7GEIHHLKZTWTGIFL2FK4KXJ2CHH2OAA, seg_pos={0 0}, upload=25807]: Done uploading piece. err: ecclient: upload failed (node:12JEJYMUS93YZ9Vxx6LT6psJmt2gn569s7dV7mBJtLsjh2e9zKE, address:104.36.82.12:4175): protocol: rpc: dial tcp 104.36.82.12:4175: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:55:32.540823312 (27h3m48.321756698s) [seg_pos={0 0}, upload=25807]: Done waiting for segment. successful: 2 optimal: 14 errs: <nil>
Jul 28 00:55:32.571298962 (27h3m48.352232329s) [upload=25807]: Done uploading segments...
Jul 28 00:55:32.571327529 (27h3m48.352260894s) [upload=25807]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 2)
Duration: 1m0.034753271s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip: 
8 out of 60 is failed (threshold=3) [....X...XX.......X...........X...................X....XX....]
duration: 
975.156257ms
FAIL (2025-07-28 00:55:30 UTC) 
orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 975.1925ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
7 out of 60 is failed (threshold=3) [.X........X.............XX...........X...X...X..............]
duration: 
401.127215ms
FAIL (2025-07-28 00:55:23 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
0}, upload=25651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 159.223.143.39:39890->104.36.82.7:4909: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:44:18.975782256 (26h52m34.756715638s) [seg_pos={0 0}, upload=25651]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 28 00:44:18.975871874 (26h52m34.756805277s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 28 00:44:18.975878645 (26h52m34.756812034s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 28 00:44:19.019856339 (26h52m34.800789708s) [upload=25651]: Done uploading segments...
Jul 28 00:44:19.019879905 (26h52m34.800813283s) [upload=25651]: Upload finished. err: <nil>
Jul 28 00:45:02.257192553 (26h53m18.03812594s) [upload=25659]: Starting upload
Jul 28 00:45:02.257306836 (26h53m18.038240216s) [upload=25659]: Done starting upload
Jul 28 00:45:02.257434360 (26h53m18.038367739s) [upload=25659]: Uploading segments...
Jul 28 00:45:02.257858888 (26h53m18.038792251s) [upload=25659]: Got next segment. Inline: false
Jul 28 00:45:02.257951856 (26h53m18.038885235s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 28 00:45:02.257959915 (26h53m18.038893293s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 28 00:45:02.293048419 (26h53m18.073981806s) [seg_pos={0 0}, upload=25659]: Begin upload segment...
Jul 28 00:45:02.293125122 (26h53m18.074058491s) [seg_pos={0 0}, upload=25659]: Done begin upload segment.
Jul 28 00:45:02.293171689 (26h53m18.074105073s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293724212 (26h53m18.074657618s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293744577 (26h53m18.074677964s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294292781 (26h53m18.075226212s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294343706 (26h53m18.075277094s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294756878 (26h53m18.075690269s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294891918 (26h53m18.075825306s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295221121 (26h53m18.076154513s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295558583 (26h53m18.076491975s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295604310 (26h53m18.0765377s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295991730 (26h53m18.07692512s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.296088221 (26h53m18.077021615s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.301695563 (26h53m18.082628952s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302203775 (26h53m18.083137164s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302629001 (26h53m18.083562389s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302995157 (26h53m18.083928524s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.418208178 (26h53m18.19914155s) [upload=25659]: Next returned nil segment
Jul 28 00:45:02.418226832 (26h53m18.19916021s) [upload=25659]: Waiting for error group managing segments...
Jul 28 00:45:02.424117363 (26h53m18.205050731s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.427234519 (26h53m18.208167896s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.430114806 (26h53m18.211048185s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.435624120 (26h53m18.216557505s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436119655 (26h53m18.217053027s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436405218 (26h53m18.217338582s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436887494 (26h53m18.217820863s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436998726 (26h53m18.217932114s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437527079 (26h53m18.218460459s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437625274 (26h53m18.218558642s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439469614 (26h53m18.220402995s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439557143 (26h53m18.220490506s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:32.221331076 (26h53m48.002264449s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: write tcp 159.223.143.39:33378->104.36.82.16:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221411781 (26h53m48.002345162s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221457184 (26h53m48.002390547s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: rpc: dial tcp 104.36.82.13:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221473904 (26h53m48.002407294s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:49546->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221516309 (26h53m48.002449671s) [seg_pos={0 0}, upload=25659]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul 28 00:45:32.251981717 (26h53m48.032915086s) [upload=25659]: Done uploading segments...
Jul 28 00:45:32.252005327 (26h53m48.032938704s) [upload=25659]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 401.176205ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip: 
8 out of 60 is failed (threshold=3) [.....X...XX.......X...........X...................X....XX...]
duration: 
368.276625ms
FAIL (2025-07-28 00:54:35 UTC) 
orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 368.370982ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)
antiblip: 
48 out of 60 is failed (threshold=3) [...........XXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration: 
1m0.037660361s
FAIL (2025-07-28 00:54:32 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 1)
======LOGS======
O2G3MLJ4BHHOTL6KI7DNJENI67FFU47LWASZIGBBEUX5A, seg_pos={0 0}, upload=25794]: Uploading piece...
Jul 28 00:53:32.514770445 (27h1m48.295703832s) [noise=true , address=104.36.82.53:4151    , piece_id=H2FQ2LIL2NOOQLDF6CTCK7MOLHGQ2XIP6JAYNQW26LLSRTDEYUWA, seg_pos={0 0}, upload=25794]: Uploading piece...
Jul 28 00:53:32.515384471 (27h1m48.296317854s) [noise=true , address=104.36.82.9:4008     , piece_id=QGJXLOOLTPQ7CASBXFLKEGJ6KLJWLWA5RCA3TMWDBHDAL7EOPDXA, seg_pos={0 0}, upload=25794]: Uploading piece...
Jul 28 00:53:32.516838401 (27h1m48.297771787s) [noise=true , address=104.36.82.15:4267    , piece_id=RZVXOJRA62DYPXO4DPON4RPJB72ZHIN3E6CEL4LZS7KAMACBHY4A, seg_pos={0 0}, upload=25794]: Uploading piece...
Jul 28 00:53:32.518746950 (27h1m48.299680343s) [noise=true , address=104.36.82.2:4762     , piece_id=IS4DLXJQWUKR64GRO5GRRPOMNJ7DRQ57ENUQIHU7MSV7GIDCISAQ, seg_pos={0 0}, upload=25794]: Uploading piece...
Jul 28 00:53:32.520509758 (27h1m48.301443143s) [noise=true , address=104.36.82.12:4166    , piece_id=GE6W5FRPQ6OGFGA4RZQR5SNZ5OIKNJPSPZ7WDWSUOSSDL5FY3YSQ, seg_pos={0 0}, upload=25794]: Uploading piece...
Jul 28 00:53:32.521121111 (27h1m48.302054497s) [noise=true , address=104.36.82.42:4827    , piece_id=ZYYEIYUVDKQYYLCDM4BLCX6VSTZB2WZEZRJYTJIZY2CMR4GLF6NQ, seg_pos={0 0}, upload=25794]: Uploading piece...
Jul 28 00:53:32.739290625 (27h1m48.520223994s) [upload=25794]: Next returned nil segment
Jul 28 00:53:32.739308344 (27h1m48.52024172s) [upload=25794]: Waiting for error group managing segments...
Jul 28 00:53:37.658699916 (27h1m53.439633285s) [noise=true , address=79.127.201.215:4348  , piece_id=FE5MDVS5FKBCL7PPR76M23RYGVKW2O3EKMY7H6BRLN2EM4TI7C5A, seg_pos={0 0}, upload=25794]: Done uploading piece. err: <nil>
Jul 28 00:54:32.437486552 (27h2m48.218419931s) [noise=true , address=104.36.82.2:4762     , piece_id=IS4DLXJQWUKR64GRO5GRRPOMNJ7DRQ57ENUQIHU7MSV7GIDCISAQ, seg_pos={0 0}, upload=25794]: Done uploading piece. err: ecclient: upload failed (node:12r1mav9KnS7y2V5XxeDexYM4Gv9JcfmqNbML5pwSEyzCZFvJra, address:104.36.82.2:4762): protocol: rpc: dial tcp 104.36.82.2:4762: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:54:32.437518541 (27h2m48.218451919s) [noise=true , address=104.36.82.9:4008     , piece_id=QGJXLOOLTPQ7CASBXFLKEGJ6KLJWLWA5RCA3TMWDBHDAL7EOPDXA, seg_pos={0 0}, upload=25794]: Done uploading piece. err: ecclient: upload failed (node:148CqQmrsJSfoGByHYRPoZmvP1VgQQpaMVLb7RgRQvNBuo7air, address:104.36.82.9:4008): protocol: rpc: dial tcp 104.36.82.9:4008: i/o timeout; context deadline exceeded
Jul 28 00:54:32.438264729 (27h2m48.219198106s) [noise=true , address=104.36.82.12:4166    , piece_id=GE6W5FRPQ6OGFGA4RZQR5SNZ5OIKNJPSPZ7WDWSUOSSDL5FY3YSQ, seg_pos={0 0}, upload=25794]: Done uploading piece. err: ecclient: upload failed (node:12iCB7c4ZfpqafpD1PtpbE6BfU81wJW65LmrPVJXtD58yeuoR9t, address:104.36.82.12:4166): protocol: rpc: dial tcp 104.36.82.12:4166: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:54:32.438381367 (27h2m48.219314742s) [noise=true , address=104.36.82.42:4827    , piece_id=ZYYEIYUVDKQYYLCDM4BLCX6VSTZB2WZEZRJYTJIZY2CMR4GLF6NQ, seg_pos={0 0}, upload=25794]: Done uploading piece. err: ecclient: upload failed (node:1pqAbf2u3J689bYTQUaaxWdF4R241i7qYHraSWK14nV68pSUfC, address:104.36.82.42:4827): protocol: write tcp 159.223.143.39:40806->104.36.82.42:4827: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:54:32.438451837 (27h2m48.219385213s) [noise=true , address=104.36.82.50:4034    , piece_id=262RJ6WISTRQHOQYLZYG7HXD6XD4Q5HOCY7U7KOIERGV35K7TEXA, seg_pos={0 0}, upload=25794]: Done uploading piece. err: ecclient: upload failed (node:12q2ocJMc2bzMsrFH4heg52ZBziCHNxQc7cxDGU79AeUZxqoB9r, address:104.36.82.50:4034): protocol: write tcp 159.223.143.39:52312->104.36.82.50:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:54:32.438549535 (27h2m48.219482911s) [noise=true , address=104.36.82.4:4808     , piece_id=YN6YFQFOY2ECYQSA4UWE5QPKL3YBTBXRHMZEUNORB7IBTWVTIVNA, seg_pos={0 0}, upload=25794]: Done uploading piece. err: ecclient: upload failed (node:1fbQCaGY9ZssBzyNusnhyY994e8ytvW4sQDQRt7q2tdGrg6ps3, address:104.36.82.4:4808): protocol: rpc: dial tcp 104.36.82.4:4808: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:54:32.438611569 (27h2m48.219544945s) [noise=true , address=104.36.82.53:4123    , piece_id=CSU3ZALVNTEMUKJAXWKZ3AXR4XSKVDFJ2JFJHPL64HW7I3VIPSWQ, seg_pos={0 0}, upload=25794]: Done uploading piece. err: ecclient: upload failed (node:12ttjngUh34tGJpCBgVCLfqrWwsdmd88XKWPrmaJrgkMu8htkKM, address:104.36.82.53:4123): protocol: write tcp 159.223.143.39:34236->104.36.82.53:4123: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:54:32.438676376 (27h2m48.219609751s) [noise=true , address=104.36.82.15:4267    , piece_id=RZVXOJRA62DYPXO4DPON4RPJB72ZHIN3E6CEL4LZS7KAMACBHY4A, seg_pos={0 0}, upload=25794]: Done uploading piece. err: ecclient: upload failed (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7, address:104.36.82.15:4267): protocol: write tcp 159.223.143.39:44082->104.36.82.15:4267: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:54:32.438702829 (27h2m48.219636207s) [noise=true , address=104.36.82.3:4797     , piece_id=T35Z5NKTNSHPK72T2X2JY5M7EHAXFU2HONHNKJMK7HXG3ZQSDCLA, seg_pos={0 0}, upload=25794]: Done uploading piece. err: ecclient: upload failed (node:1uC4tCyRnreNPqyXpsU6K8rPqShbaRHRgNX6amTf6aL7j23rQu, address:104.36.82.3:4797): protocol: write tcp 159.223.143.39:40956->104.36.82.3:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:54:32.438773197 (27h2m48.219706572s) [noise=true , address=104.36.82.52:4101    , piece_id=YRDT7ZZUOA4F6ZXNELRWMU2A7CQJBO2BD5SHM6BLOOVPH4VZLT3Q, seg_pos={0 0}, upload=25794]: Done uploading piece. err: ecclient: upload failed (node:12WWXeBjFZR9PzciNfs35fFcx87N9GwDpMsPHnksFDugtaBGWp4, address:104.36.82.52:4101): protocol: write tcp 159.223.143.39:33032->104.36.82.52:4101: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:54:32.438778433 (27h2m48.219711804s) [noise=true , address=104.36.82.47:4517    , piece_id=TQNCA5TO2G3MLJ4BHHOTL6KI7DNJENI67FFU47LWASZIGBBEUX5A, seg_pos={0 0}, upload=25794]: Done uploading piece. err: ecclient: upload failed (node:12HZjKTnjpeNU5B6muVHGrmWQS8GsyuYTW6cLxzNk5CGt3b8PJS, address:104.36.82.47:4517): protocol: rpc: dial tcp 104.36.82.47:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:54:32.438839230 (27h2m48.219772606s) [noise=true , address=104.36.82.54:4160    , piece_id=BBCRZHRWAGFWOEIKCUKVOEPJAH76X6Z4DCBAK6FR6U6LI6TGVD3A, seg_pos={0 0}, upload=25794]: Done uploading piece. err: ecclient: upload failed (node:12qRsRx7vmb3S9m5J7gceK44gnuuYYMyorSAyxcpjpPeRJJFZ84, address:104.36.82.54:4160): protocol: write tcp 159.223.143.39:38696->104.36.82.54:4160: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:54:32.438873505 (27h2m48.219806873s) [noise=true , address=104.36.82.53:4151    , piece_id=H2FQ2LIL2NOOQLDF6CTCK7MOLHGQ2XIP6JAYNQW26LLSRTDEYUWA, seg_pos={0 0}, upload=25794]: Done uploading piece. err: ecclient: upload failed (node:196fMe3fbxKaPgNs1zQx7Kg3Rh68s6Gtp7mbu3hn654WCbZctx, address:104.36.82.53:4151): protocol: write tcp 159.223.143.39:43504->104.36.82.53:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:54:32.438884024 (27h2m48.219817391s) [noise=true , address=104.36.82.10:4082    , piece_id=BNYABHXB536EBUIFG5W6LS5CVHRA36J353GKZGRUWHY3AL265DLA, seg_pos={0 0}, upload=25794]: Done uploading piece. err: ecclient: upload failed (node:12DKsNCtqxLvpJwLszdLs74V3eb8NH2VAV9V3bngzUczawmNoLK, address:104.36.82.10:4082): protocol: write tcp 159.223.143.39:60888->104.36.82.10:4082: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:54:32.438898863 (27h2m48.219832338s) [noise=true , address=104.36.82.42:4809    , piece_id=QDOZHHXPBLXYGCX4XLMO3GG65WENZBV3D5X22VTVO72E575YVPSQ, seg_pos={0 0}, upload=25794]: Done uploading piece. err: ecclient: upload failed (node:1VZgucaa2gRuFhNxARMb7BDS6EGZXKChGPppbFksbPc5we7fTQ, address:104.36.82.42:4809): protocol: write tcp 159.223.143.39:47810->104.36.82.42:4809: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:54:32.438909985 (27h2m48.219843366s) [noise=true , address=104.36.82.5:4860     , piece_id=7DDHEYOPNHT5RR2KLJ3EIMYESGLWHSTBOPGEXANXF33KM23K5XQQ, seg_pos={0 0}, upload=25794]: Done uploading piece. err: ecclient: upload failed (node:1Qr9fghoRVbAvNm9AMEHhhtc7JqHWBuZerBttg3KW3J3GcDjrE, address:104.36.82.5:4860): protocol: rpc: dial tcp 104.36.82.5:4860: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:54:32.438935820 (27h2m48.219869209s) [noise=true , address=104.36.82.44:4902    , piece_id=FZGBKOO7234IWOQD4LGBRVQJGFCOC5V7TDDSMNA6HCN6ZEAKI77Q, seg_pos={0 0}, upload=25794]: Done uploading piece. err: ecclient: upload failed (node:1Be9T5kXUwLU3C6wBeKFV3Q7aZFbZDTNG1oaEft8V6rmySUcEE, address:104.36.82.44:4902): protocol: write tcp 159.223.143.39:36176->104.36.82.44:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:54:32.438979340 (27h2m48.219912724s) [seg_pos={0 0}, upload=25794]: Done waiting for segment. successful: 1 optimal: 14 errs: <nil>
Jul 28 00:54:32.472283066 (27h2m48.253216436s) [upload=25794]: Done uploading segments...
Jul 28 00:54:32.472317288 (27h2m48.253250667s) [upload=25794]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 1)
Duration: 1m0.037694088s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
7 out of 60 is failed (threshold=3) [..X........X.............XX...........X...X...X.............]
duration: 
452.673593ms
FAIL (2025-07-28 00:54:19 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
0}, upload=25651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 159.223.143.39:39890->104.36.82.7:4909: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:44:18.975782256 (26h52m34.756715638s) [seg_pos={0 0}, upload=25651]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 28 00:44:18.975871874 (26h52m34.756805277s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 28 00:44:18.975878645 (26h52m34.756812034s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 28 00:44:19.019856339 (26h52m34.800789708s) [upload=25651]: Done uploading segments...
Jul 28 00:44:19.019879905 (26h52m34.800813283s) [upload=25651]: Upload finished. err: <nil>
Jul 28 00:45:02.257192553 (26h53m18.03812594s) [upload=25659]: Starting upload
Jul 28 00:45:02.257306836 (26h53m18.038240216s) [upload=25659]: Done starting upload
Jul 28 00:45:02.257434360 (26h53m18.038367739s) [upload=25659]: Uploading segments...
Jul 28 00:45:02.257858888 (26h53m18.038792251s) [upload=25659]: Got next segment. Inline: false
Jul 28 00:45:02.257951856 (26h53m18.038885235s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 28 00:45:02.257959915 (26h53m18.038893293s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 28 00:45:02.293048419 (26h53m18.073981806s) [seg_pos={0 0}, upload=25659]: Begin upload segment...
Jul 28 00:45:02.293125122 (26h53m18.074058491s) [seg_pos={0 0}, upload=25659]: Done begin upload segment.
Jul 28 00:45:02.293171689 (26h53m18.074105073s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293724212 (26h53m18.074657618s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293744577 (26h53m18.074677964s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294292781 (26h53m18.075226212s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294343706 (26h53m18.075277094s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294756878 (26h53m18.075690269s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294891918 (26h53m18.075825306s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295221121 (26h53m18.076154513s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295558583 (26h53m18.076491975s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295604310 (26h53m18.0765377s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295991730 (26h53m18.07692512s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.296088221 (26h53m18.077021615s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.301695563 (26h53m18.082628952s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302203775 (26h53m18.083137164s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302629001 (26h53m18.083562389s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302995157 (26h53m18.083928524s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.418208178 (26h53m18.19914155s) [upload=25659]: Next returned nil segment
Jul 28 00:45:02.418226832 (26h53m18.19916021s) [upload=25659]: Waiting for error group managing segments...
Jul 28 00:45:02.424117363 (26h53m18.205050731s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.427234519 (26h53m18.208167896s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.430114806 (26h53m18.211048185s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.435624120 (26h53m18.216557505s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436119655 (26h53m18.217053027s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436405218 (26h53m18.217338582s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436887494 (26h53m18.217820863s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436998726 (26h53m18.217932114s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437527079 (26h53m18.218460459s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437625274 (26h53m18.218558642s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439469614 (26h53m18.220402995s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439557143 (26h53m18.220490506s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:32.221331076 (26h53m48.002264449s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: write tcp 159.223.143.39:33378->104.36.82.16:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221411781 (26h53m48.002345162s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221457184 (26h53m48.002390547s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: rpc: dial tcp 104.36.82.13:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221473904 (26h53m48.002407294s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:49546->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221516309 (26h53m48.002449671s) [seg_pos={0 0}, upload=25659]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul 28 00:45:32.251981717 (26h53m48.032915086s) [upload=25659]: Done uploading segments...
Jul 28 00:45:32.252005327 (26h53m48.032938704s) [upload=25659]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 452.720904ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
7 out of 60 is failed (threshold=3) [...X........X.............XX...........X...X...X............]
duration: 
350.640016ms
FAIL (2025-07-28 00:53:55 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
0}, upload=25651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 159.223.143.39:39890->104.36.82.7:4909: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:44:18.975782256 (26h52m34.756715638s) [seg_pos={0 0}, upload=25651]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 28 00:44:18.975871874 (26h52m34.756805277s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 28 00:44:18.975878645 (26h52m34.756812034s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 28 00:44:19.019856339 (26h52m34.800789708s) [upload=25651]: Done uploading segments...
Jul 28 00:44:19.019879905 (26h52m34.800813283s) [upload=25651]: Upload finished. err: <nil>
Jul 28 00:45:02.257192553 (26h53m18.03812594s) [upload=25659]: Starting upload
Jul 28 00:45:02.257306836 (26h53m18.038240216s) [upload=25659]: Done starting upload
Jul 28 00:45:02.257434360 (26h53m18.038367739s) [upload=25659]: Uploading segments...
Jul 28 00:45:02.257858888 (26h53m18.038792251s) [upload=25659]: Got next segment. Inline: false
Jul 28 00:45:02.257951856 (26h53m18.038885235s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 28 00:45:02.257959915 (26h53m18.038893293s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 28 00:45:02.293048419 (26h53m18.073981806s) [seg_pos={0 0}, upload=25659]: Begin upload segment...
Jul 28 00:45:02.293125122 (26h53m18.074058491s) [seg_pos={0 0}, upload=25659]: Done begin upload segment.
Jul 28 00:45:02.293171689 (26h53m18.074105073s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293724212 (26h53m18.074657618s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293744577 (26h53m18.074677964s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294292781 (26h53m18.075226212s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294343706 (26h53m18.075277094s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294756878 (26h53m18.075690269s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294891918 (26h53m18.075825306s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295221121 (26h53m18.076154513s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295558583 (26h53m18.076491975s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295604310 (26h53m18.0765377s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295991730 (26h53m18.07692512s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.296088221 (26h53m18.077021615s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.301695563 (26h53m18.082628952s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302203775 (26h53m18.083137164s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302629001 (26h53m18.083562389s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302995157 (26h53m18.083928524s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.418208178 (26h53m18.19914155s) [upload=25659]: Next returned nil segment
Jul 28 00:45:02.418226832 (26h53m18.19916021s) [upload=25659]: Waiting for error group managing segments...
Jul 28 00:45:02.424117363 (26h53m18.205050731s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.427234519 (26h53m18.208167896s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.430114806 (26h53m18.211048185s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.435624120 (26h53m18.216557505s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436119655 (26h53m18.217053027s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436405218 (26h53m18.217338582s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436887494 (26h53m18.217820863s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436998726 (26h53m18.217932114s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437527079 (26h53m18.218460459s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437625274 (26h53m18.218558642s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439469614 (26h53m18.220402995s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439557143 (26h53m18.220490506s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:32.221331076 (26h53m48.002264449s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: write tcp 159.223.143.39:33378->104.36.82.16:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221411781 (26h53m48.002345162s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221457184 (26h53m48.002390547s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: rpc: dial tcp 104.36.82.13:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221473904 (26h53m48.002407294s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:49546->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221516309 (26h53m48.002449671s) [seg_pos={0 0}, upload=25659]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul 28 00:45:32.251981717 (26h53m48.032915086s) [upload=25659]: Done uploading segments...
Jul 28 00:45:32.252005327 (26h53m48.032938704s) [upload=25659]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 350.669765ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)
antiblip: 
47 out of 60 is failed (threshold=3) [............XXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration: 
1m0.02756583s
FAIL (2025-07-28 00:53:32 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 2)
======LOGS======
upload=25776]: Uploading piece...
Jul 28 00:52:32.425968696 (27h0m48.206902079s) [noise=true , address=79.127.201.215:4343  , piece_id=FWA32LWPCODWQ3BCJ33GBG6RUIEHOHRG4XH7IAFNK2F6JT7Y2JRQ, seg_pos={0 0}, upload=25776]: Uploading piece...
Jul 28 00:52:32.425993216 (27h0m48.206926595s) [noise=true , address=104.36.82.10:4100    , piece_id=UINOD7YMCNVF5M4XR7HLHU7GELQYKZG4XEAASI3Z5DC33BNGWEWQ, seg_pos={0 0}, upload=25776]: Uploading piece...
Jul 28 00:52:32.426451123 (27h0m48.2073845s) [noise=true , address=104.36.82.12:4175    , piece_id=3AL6UDK7UCTSFAJA3W2M7FHPYBCBK32NGRGDMXBOOJYDG637JTFA, seg_pos={0 0}, upload=25776]: Uploading piece...
Jul 28 00:52:32.426505916 (27h0m48.207439302s) [noise=true , address=104.36.82.51:4074    , piece_id=C2K7YY5NHKN2TA7RTSHCG5WEOMBURQ4QYXQ7A3GUUV6XCAQGT2LQ, seg_pos={0 0}, upload=25776]: Uploading piece...
Jul 28 00:52:32.427481828 (27h0m48.208415211s) [noise=true , address=104.36.82.22:4521    , piece_id=NQWY6FW4WPR2TXAAKEFWO3FP67L6Z6Y22IMSYLJF6BDVLYPWP7RQ, seg_pos={0 0}, upload=25776]: Uploading piece...
Jul 28 00:52:32.428439452 (27h0m48.209372837s) [noise=true , address=104.36.82.53:4145    , piece_id=ZNPU276GPTBKUNQSEL5DI6AM7OZU22WJ2XLLWPGIBLCUOCLWIFDQ, seg_pos={0 0}, upload=25776]: Uploading piece...
Jul 28 00:52:32.429526074 (27h0m48.210459438s) [noise=true , address=104.36.82.52:4115    , piece_id=A66DFG2UL4UUNRIUKTPPGGART32JWPMTCDTNPTPWW4S3MLSQIUHQ, seg_pos={0 0}, upload=25776]: Uploading piece...
Jul 28 00:52:32.639664292 (27h0m48.420597661s) [upload=25776]: Next returned nil segment
Jul 28 00:52:32.639681443 (27h0m48.42061482s) [upload=25776]: Waiting for error group managing segments...
Jul 28 00:52:33.135568939 (27h0m48.916502315s) [noise=true , address=79.127.205.245:4251  , piece_id=MSBW4AR56NM2V35GNWXUYPPNY3QUUYKET4UXZKHXO4TNBNM6Q4QQ, seg_pos={0 0}, upload=25776]: Done uploading piece. err: <nil>
Jul 28 00:52:35.411579885 (27h0m51.192513259s) [noise=true , address=79.127.201.215:4343  , piece_id=FWA32LWPCODWQ3BCJ33GBG6RUIEHOHRG4XH7IAFNK2F6JT7Y2JRQ, seg_pos={0 0}, upload=25776]: Done uploading piece. err: <nil>
Jul 28 00:53:32.345723698 (27h1m48.126657067s) [noise=true , address=104.36.82.13:4215    , piece_id=VB54HXUU4TQ2DTKVHHLR3ID34GNFHFJNUBYBKWBZD7Y3IWDT546Q, seg_pos={0 0}, upload=25776]: Done uploading piece. err: ecclient: upload failed (node:1vV1hGhZqeEoh5ei7yRu49oyAvTZ2T7S1x3ooufJPtFH2TGBkQ, address:104.36.82.13:4215): protocol: write tcp 159.223.143.39:48416->104.36.82.13:4215: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:53:32.345798055 (27h1m48.12673142s) [noise=true , address=104.36.82.56:4232    , piece_id=7OKHFJSFEQPEJMG7PX34RV46TACGGY2V2ZQSGKZA6E5XGHXUENZQ, seg_pos={0 0}, upload=25776]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:104.36.82.56:4232): protocol: write tcp 159.223.143.39:58976->104.36.82.56:4232: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:53:32.345874240 (27h1m48.126807616s) [noise=true , address=104.36.82.53:4145    , piece_id=ZNPU276GPTBKUNQSEL5DI6AM7OZU22WJ2XLLWPGIBLCUOCLWIFDQ, seg_pos={0 0}, upload=25776]: Done uploading piece. err: ecclient: upload failed (node:1rXBNYi6SgZdwGbiU5ooBE8U7DrEoAGw25ncYsRETgNh4CsiLX, address:104.36.82.53:4145): protocol: rpc: dial tcp 104.36.82.53:4145: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:53:32.345941455 (27h1m48.126874832s) [noise=true , address=104.36.82.14:4238    , piece_id=2QZFQDZZII3IUFTVBR3D2VRQIRCM7ZIOJZW54E3EN3GPR57X6EEQ, seg_pos={0 0}, upload=25776]: Done uploading piece. err: ecclient: upload failed (node:122AVjaN3YaiABnUjuZwSb6apPDeszZAWvmzwtP47k1fxsKLD5V, address:104.36.82.14:4238): protocol: write tcp 159.223.143.39:58546->104.36.82.14:4238: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:53:32.346193713 (27h1m48.127127109s) [noise=true , address=104.36.82.41:4781    , piece_id=EXXKXL5YAN3KO4OQ4THW2JIIX6GYKDGVHNH2CAIZSOHCGUDDDK4Q, seg_pos={0 0}, upload=25776]: Done uploading piece. err: ecclient: upload failed (node:1QhCkTzSogXFKLmFV4wQUcALhvm9YhCSQudLxSrvQWmepheiFi, address:104.36.82.41:4781): protocol: write tcp 159.223.143.39:51614->104.36.82.41:4781: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:53:32.346293353 (27h1m48.127226722s) [noise=true , address=104.36.82.55:4193    , piece_id=H4TU5NWNYNK4HJ5FCNZYEZ7H2PSX2LHNL4C5XWDARE3FQPLDQMEQ, seg_pos={0 0}, upload=25776]: Done uploading piece. err: ecclient: upload failed (node:1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8, address:104.36.82.55:4193): protocol: write tcp 159.223.143.39:58488->104.36.82.55:4193: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:53:32.346374596 (27h1m48.127307961s) [noise=true , address=104.36.82.10:4100    , piece_id=UINOD7YMCNVF5M4XR7HLHU7GELQYKZG4XEAASI3Z5DC33BNGWEWQ, seg_pos={0 0}, upload=25776]: Done uploading piece. err: ecclient: upload failed (node:12Hyc6sbtuPXMLyFzubMcepbJyLyVAGNpkXAmqg73i3sY97dwdW, address:104.36.82.10:4100): protocol: rpc: dial tcp 104.36.82.10:4100: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:53:32.346441033 (27h1m48.127374399s) [noise=true , address=104.36.82.51:4074    , piece_id=C2K7YY5NHKN2TA7RTSHCG5WEOMBURQ4QYXQ7A3GUUV6XCAQGT2LQ, seg_pos={0 0}, upload=25776]: Done uploading piece. err: ecclient: upload failed (node:1DBQpuE1EC1ENCy67We5QhMT6zsa2AbeMDSv8hmQLjokGRvoxc, address:104.36.82.51:4074): protocol: write tcp 159.223.143.39:39566->104.36.82.51:4074: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:53:32.346618185 (27h1m48.127551553s) [noise=true , address=104.36.82.54:4186    , piece_id=GRCKMQG444FMYE6MTRPRN3LYMZPTOMGQ4D75RRV3C5PGPMQY5VGQ, seg_pos={0 0}, upload=25776]: Done uploading piece. err: ecclient: upload failed (node:12Gu3vXy2JUkKDaF4MkMCgVmYspW4Ez8EAfv6e4GeCkrwYTe5T, address:104.36.82.54:4186): protocol: write tcp 159.223.143.39:34922->104.36.82.54:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:53:32.346677592 (27h1m48.127610957s) [noise=true , address=104.36.82.11:4144    , piece_id=ZQWNF6ME6B3UAT5OOIFTO77CAMUL6KTF4E5AOGZMN2H3XQF24XWA, seg_pos={0 0}, upload=25776]: Done uploading piece. err: ecclient: upload failed (node:128246NkJjT3UFiQdSqFssY6jTAV1jye3dXAjAkFnC3DUAXb235, address:104.36.82.11:4144): protocol: write tcp 159.223.143.39:59954->104.36.82.11:4144: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:53:32.346728163 (27h1m48.127661528s) [noise=true , address=104.36.82.50:4013    , piece_id=FAQ3ZO7QJLPKEKG5XQ555ESOGN32KM5RXG44BFLCTC7FP423GTDA, seg_pos={0 0}, upload=25776]: Done uploading piece. err: ecclient: upload failed (node:121scLvKEW9zw2aEz22zpremcjiop67eQ8v2uqpmaD5aZzCAP7b, address:104.36.82.50:4013): protocol: rpc: dial tcp 104.36.82.50:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:53:32.346777048 (27h1m48.127710423s) [noise=true , address=104.36.82.42:4818    , piece_id=TETHZWUROAZRNKQVWNMXC653NER7TJP5A5AK5S2XK57EWGZBPEUQ, seg_pos={0 0}, upload=25776]: Done uploading piece. err: ecclient: upload failed (node:122m1dk7cCwZfUKeq4BChF1UjXSt7swvt1L3HdiSybavXhLEcsb, address:104.36.82.42:4818): protocol: write tcp 159.223.143.39:39804->104.36.82.42:4818: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:53:32.346818676 (27h1m48.127752049s) [noise=true , address=104.36.82.21:4480    , piece_id=V6F74FAOSY6JQYW2GQNWFARD3JW3NAA2XKFNI7SLHVY34R2U54VA, seg_pos={0 0}, upload=25776]: Done uploading piece. err: ecclient: upload failed (node:12qAtwKs88zrSjvRR9fvez9J9HnRBF2Ax7APfCq9BLHSFTE24a6, address:104.36.82.21:4480): protocol: write tcp 159.223.143.39:37400->104.36.82.21:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:53:32.346879796 (27h1m48.127813161s) [noise=true , address=104.36.82.12:4175    , piece_id=3AL6UDK7UCTSFAJA3W2M7FHPYBCBK32NGRGDMXBOOJYDG637JTFA, seg_pos={0 0}, upload=25776]: Done uploading piece. err: ecclient: upload failed (node:12JEJYMUS93YZ9Vxx6LT6psJmt2gn569s7dV7mBJtLsjh2e9zKE, address:104.36.82.12:4175): protocol: rpc: dial tcp 104.36.82.12:4175: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:53:32.346905961 (27h1m48.12783933s) [noise=true , address=104.36.82.52:4115    , piece_id=A66DFG2UL4UUNRIUKTPPGGART32JWPMTCDTNPTPWW4S3MLSQIUHQ, seg_pos={0 0}, upload=25776]: Done uploading piece. err: ecclient: upload failed (node:1ikQ2b343QztY3vBFQ3cmEwEEFszpB2TPMtstufHdFfMEkJkBV, address:104.36.82.52:4115): protocol: write tcp 159.223.143.39:50176->104.36.82.52:4115: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:53:32.346928180 (27h1m48.127861548s) [noise=true , address=104.36.82.22:4521    , piece_id=NQWY6FW4WPR2TXAAKEFWO3FP67L6Z6Y22IMSYLJF6BDVLYPWP7RQ, seg_pos={0 0}, upload=25776]: Done uploading piece. err: ecclient: upload failed (node:1zWiwpaV58h8YKsAgw24fUWHBBYYyqF4UX6GJV316Uxo9Y7gt1, address:104.36.82.22:4521): protocol: write tcp 159.223.143.39:46130->104.36.82.22:4521: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:53:32.346950455 (27h1m48.127883833s) [seg_pos={0 0}, upload=25776]: Done waiting for segment. successful: 2 optimal: 14 errs: <nil>
Jul 28 00:53:32.370720655 (27h1m48.151654029s) [upload=25776]: Done uploading segments...
Jul 28 00:53:32.370768269 (27h1m48.151701644s) [upload=25776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 2)
Duration: 1m0.027588777s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip: 
8 out of 60 is failed (threshold=3) [......X...XX.......X...........X...................X....XX..]
duration: 
385.514503ms
FAIL (2025-07-28 00:53:23 UTC) 
orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 385.542411ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
7 out of 60 is failed (threshold=3) [....X........X.............XX...........X...X...X...........]
duration: 
428.41973ms
FAIL (2025-07-28 00:53:01 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
0}, upload=25651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 159.223.143.39:39890->104.36.82.7:4909: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:44:18.975782256 (26h52m34.756715638s) [seg_pos={0 0}, upload=25651]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 28 00:44:18.975871874 (26h52m34.756805277s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 28 00:44:18.975878645 (26h52m34.756812034s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 28 00:44:19.019856339 (26h52m34.800789708s) [upload=25651]: Done uploading segments...
Jul 28 00:44:19.019879905 (26h52m34.800813283s) [upload=25651]: Upload finished. err: <nil>
Jul 28 00:45:02.257192553 (26h53m18.03812594s) [upload=25659]: Starting upload
Jul 28 00:45:02.257306836 (26h53m18.038240216s) [upload=25659]: Done starting upload
Jul 28 00:45:02.257434360 (26h53m18.038367739s) [upload=25659]: Uploading segments...
Jul 28 00:45:02.257858888 (26h53m18.038792251s) [upload=25659]: Got next segment. Inline: false
Jul 28 00:45:02.257951856 (26h53m18.038885235s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 28 00:45:02.257959915 (26h53m18.038893293s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 28 00:45:02.293048419 (26h53m18.073981806s) [seg_pos={0 0}, upload=25659]: Begin upload segment...
Jul 28 00:45:02.293125122 (26h53m18.074058491s) [seg_pos={0 0}, upload=25659]: Done begin upload segment.
Jul 28 00:45:02.293171689 (26h53m18.074105073s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293724212 (26h53m18.074657618s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293744577 (26h53m18.074677964s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294292781 (26h53m18.075226212s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294343706 (26h53m18.075277094s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294756878 (26h53m18.075690269s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294891918 (26h53m18.075825306s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295221121 (26h53m18.076154513s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295558583 (26h53m18.076491975s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295604310 (26h53m18.0765377s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295991730 (26h53m18.07692512s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.296088221 (26h53m18.077021615s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.301695563 (26h53m18.082628952s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302203775 (26h53m18.083137164s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302629001 (26h53m18.083562389s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302995157 (26h53m18.083928524s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.418208178 (26h53m18.19914155s) [upload=25659]: Next returned nil segment
Jul 28 00:45:02.418226832 (26h53m18.19916021s) [upload=25659]: Waiting for error group managing segments...
Jul 28 00:45:02.424117363 (26h53m18.205050731s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.427234519 (26h53m18.208167896s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.430114806 (26h53m18.211048185s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.435624120 (26h53m18.216557505s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436119655 (26h53m18.217053027s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436405218 (26h53m18.217338582s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436887494 (26h53m18.217820863s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436998726 (26h53m18.217932114s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437527079 (26h53m18.218460459s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437625274 (26h53m18.218558642s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439469614 (26h53m18.220402995s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439557143 (26h53m18.220490506s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:32.221331076 (26h53m48.002264449s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: write tcp 159.223.143.39:33378->104.36.82.16:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221411781 (26h53m48.002345162s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221457184 (26h53m48.002390547s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: rpc: dial tcp 104.36.82.13:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221473904 (26h53m48.002407294s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:49546->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221516309 (26h53m48.002449671s) [seg_pos={0 0}, upload=25659]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul 28 00:45:32.251981717 (26h53m48.032915086s) [upload=25659]: Done uploading segments...
Jul 28 00:45:32.252005327 (26h53m48.032938704s) [upload=25659]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 428.450058ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip: 
8 out of 60 is failed (threshold=3) [.......X...XX.......X...........X...................X....XX.]
duration: 
503.584138ms
FAIL (2025-07-28 00:52:59 UTC) 
orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 503.68467ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip: 
8 out of 60 is failed (threshold=3) [........X...XX.......X...........X...................X....XX]
duration: 
30.051106343s
FAIL (2025-07-28 00:52:51 UTC) 
orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 30.051189345s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)
antiblip: 
46 out of 60 is failed (threshold=3) [.............XXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration: 
1m0.033341494s
FAIL (2025-07-28 00:52:32 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 1)
======LOGS======
ing piece...
Jul 28 00:51:32.310240927 (26h59m48.091174302s) [noise=true , address=104.36.82.46:4497    , piece_id=3OJ2APUS7M3P6JSLTARCQXF6QJU66JQXH333USGMGD3G53CWQYKA, seg_pos={0 0}, upload=25760]: Uploading piece...
Jul 28 00:51:32.310623663 (26h59m48.091557039s) [noise=true , address=104.36.82.47:4549    , piece_id=P52UCEDCBEDYYFFRFRBN3ZM77PD4OV7HK3JWSQDHFQ3FJW3M4NEA, seg_pos={0 0}, upload=25760]: Uploading piece...
Jul 28 00:51:32.311077490 (26h59m48.092010865s) [noise=true , address=104.36.82.38:4704    , piece_id=YH5BYISJPEWLCGFI5DAZNHMC47TJSWS4A57TN7ZTKCIOZOAW4GMA, seg_pos={0 0}, upload=25760]: Uploading piece...
Jul 28 00:51:32.311501310 (26h59m48.092434689s) [noise=true , address=104.36.82.6:4906     , piece_id=37AGK7YMIZVX2EJ62OPMQSEKEHQYFOXXIK3SZOLUS4RMU7XST6HQ, seg_pos={0 0}, upload=25760]: Uploading piece...
Jul 28 00:51:32.311848168 (26h59m48.092781543s) [noise=true , address=104.36.82.15:4268    , piece_id=FSQRXXT5SO273BIDRKQEAV3B5S6Y7BIZ22KVG3OFSQOX6QCMZWBQ, seg_pos={0 0}, upload=25760]: Uploading piece...
Jul 28 00:51:32.511341250 (26h59m48.292274646s) [upload=25760]: Next returned nil segment
Jul 28 00:51:32.511906931 (26h59m48.292840312s) [upload=25760]: Waiting for error group managing segments...
Jul 28 00:51:32.894813850 (26h59m48.675747213s) [noise=true , address=79.127.201.215:4343  , piece_id=VD5G5VDZYQCFQ4OCSTVSI37OHZ5CLNDVRAEKJLIVEAZYHHLJPT5Q, seg_pos={0 0}, upload=25760]: Done uploading piece. err: <nil>
Jul 28 00:52:32.240344951 (27h0m48.021278328s) [noise=true , address=104.36.82.54:4160    , piece_id=KW7PMRGH3WDP4K7DLKUN44W7ZTM7ODWF23O5AKYKTKVVQJBSGOXA, seg_pos={0 0}, upload=25760]: Done uploading piece. err: ecclient: upload failed (node:12qRsRx7vmb3S9m5J7gceK44gnuuYYMyorSAyxcpjpPeRJJFZ84, address:104.36.82.54:4160): protocol: write tcp 159.223.143.39:59412->104.36.82.54:4160: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:52:32.240425124 (27h0m48.021358502s) [noise=true , address=104.36.82.20:4444    , piece_id=KWEITJIEGJURFOKBEEYDV3EN7W7BKLXOIAKXYHLZTA2E33QDRFKA, seg_pos={0 0}, upload=25760]: Done uploading piece. err: ecclient: upload failed (node:12m5PditUsvMEejqnuEBZ6jLunbkTw9dEkmGzLcBg4y2XAwhBTD, address:104.36.82.20:4444): protocol: write tcp 159.223.143.39:45388->104.36.82.20:4444: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:52:32.240488212 (27h0m48.021421588s) [noise=true , address=104.36.82.6:4906     , piece_id=37AGK7YMIZVX2EJ62OPMQSEKEHQYFOXXIK3SZOLUS4RMU7XST6HQ, seg_pos={0 0}, upload=25760]: Done uploading piece. err: ecclient: upload failed (node:1gByS6jTUkLuHrY8MafgmkmB4wXyY2j9Mt48fkpPNWbCKTcnty, address:104.36.82.6:4906): protocol: write tcp 159.223.143.39:46734->104.36.82.6:4906: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:52:32.240657189 (27h0m48.021590572s) [noise=true , address=104.36.82.52:4093    , piece_id=OJQISEZBSEKMBJND24MZDR5NNLHBOQ6KWEXZCA2XLIDROMHGLGIA, seg_pos={0 0}, upload=25760]: Done uploading piece. err: ecclient: upload failed (node:12APDQdo9gz4mqC7TxxZE6zbf92LLP26iUYfpcpri4LyQn2ic4E, address:104.36.82.52:4093): protocol: write tcp 159.223.143.39:43334->104.36.82.52:4093: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:52:32.240768472 (27h0m48.021701848s) [noise=true , address=104.36.82.12:4170    , piece_id=WK7QMQZG4HT54A3CVQ42CNII4Z6AT2GCJH6E3K3SISK7EMEQW3HQ, seg_pos={0 0}, upload=25760]: Done uploading piece. err: ecclient: upload failed (node:1EfVDsPfsWymV1yfufQw2BidpCGqqqsmtrvoid67a5ug9YYyyd, address:104.36.82.12:4170): protocol: write tcp 159.223.143.39:43402->104.36.82.12:4170: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:52:32.240876681 (27h0m48.021810059s) [noise=true , address=104.36.82.11:4144    , piece_id=4GFWGWEPM726OJCEQMM2UHOLT3QGB4ZACOHEL5IB5COY3DE77PGQ, seg_pos={0 0}, upload=25760]: Done uploading piece. err: ecclient: upload failed (node:128246NkJjT3UFiQdSqFssY6jTAV1jye3dXAjAkFnC3DUAXb235, address:104.36.82.11:4144): protocol: write tcp 159.223.143.39:58910->104.36.82.11:4144: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:52:32.240971229 (27h0m48.021904609s) [noise=true , address=104.36.82.55:4222    , piece_id=Z6MR2NQIUVYWP3ENRJKRH46P2SAHSEHVWPT3ZXEN4D2AHNNQX6IQ, seg_pos={0 0}, upload=25760]: Done uploading piece. err: ecclient: upload failed (node:12ZKW35KvrsoYRefbYeUFZo8bh4XZYycNKGEvTFjzibezjXgj3e, address:104.36.82.55:4222): protocol: write tcp 159.223.143.39:39842->104.36.82.55:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:52:32.241136890 (27h0m48.022070268s) [noise=true , address=104.36.82.38:4704    , piece_id=YH5BYISJPEWLCGFI5DAZNHMC47TJSWS4A57TN7ZTKCIOZOAW4GMA, seg_pos={0 0}, upload=25760]: Done uploading piece. err: ecclient: upload failed (node:1225CoSQvGuiWjJZWaXhvvq4gxzMymZqUb9fBrfLWw4RekwAEhp, address:104.36.82.38:4704): protocol: write tcp 159.223.143.39:37730->104.36.82.38:4704: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:52:32.241210383 (27h0m48.022143765s) [noise=true , address=104.36.82.44:4892    , piece_id=JDOF6HP273QAOCW7UNNMK5ZOXVTS5K3DIVBXZCQ6KRSMJ7I6SV2Q, seg_pos={0 0}, upload=25760]: Done uploading piece. err: ecclient: upload failed (node:12RHwFxcUjPsn12uHEAH3fBDJKv1v1tM8mRJXoFrZrG5djdswF5, address:104.36.82.44:4892): protocol: write tcp 159.223.143.39:42076->104.36.82.44:4892: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:52:32.241279257 (27h0m48.022212634s) [noise=true , address=104.36.82.15:4268    , piece_id=FSQRXXT5SO273BIDRKQEAV3B5S6Y7BIZ22KVG3OFSQOX6QCMZWBQ, seg_pos={0 0}, upload=25760]: Done uploading piece. err: ecclient: upload failed (node:112ocg8g8hf2huUskC1Kty3z51wXc8VuNUgARTFjKZxsF6RFFy, address:104.36.82.15:4268): protocol: write tcp 159.223.143.39:46458->104.36.82.15:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:52:32.241341892 (27h0m48.022275271s) [noise=true , address=104.36.82.56:4247    , piece_id=7CIUS3B7WRZOARLMDV4GPVMMVOBHKMFQD2T2H4TUOCV57UU5AFAQ, seg_pos={0 0}, upload=25760]: Done uploading piece. err: ecclient: upload failed (node:12PzKreuPT9X594ZTTYbXanNdaFVVJsnKL99fumjXymnRa9jeEq, address:104.36.82.56:4247): protocol: write tcp 159.223.143.39:55442->104.36.82.56:4247: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:52:32.241618609 (27h0m48.022551993s) [noise=true , address=104.36.82.47:4549    , piece_id=P52UCEDCBEDYYFFRFRBN3ZM77PD4OV7HK3JWSQDHFQ3FJW3M4NEA, seg_pos={0 0}, upload=25760]: Done uploading piece. err: ecclient: upload failed (node:1HWoyMKQ6v2XWUfEmfFh3m8AiGivAcYkkZ9na8kqHFzH2hPDeK, address:104.36.82.47:4549): protocol: write tcp 159.223.143.39:48226->104.36.82.47:4549: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:52:32.241704353 (27h0m48.022637729s) [noise=true , address=104.36.82.46:4497    , piece_id=3OJ2APUS7M3P6JSLTARCQXF6QJU66JQXH333USGMGD3G53CWQYKA, seg_pos={0 0}, upload=25760]: Done uploading piece. err: ecclient: upload failed (node:12Zg4PSYBuLxFvF9oYiuvn65JNbjT5RdagcVAQLp2izvjPe1NEc, address:104.36.82.46:4497): protocol: write tcp 159.223.143.39:40440->104.36.82.46:4497: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:52:32.241718098 (27h0m48.022651479s) [noise=true , address=104.36.82.53:4122    , piece_id=35THZZXRKBZFC3MV26YOP67GQIBCMQVJSYPA732Z5WKIMDJVLYVA, seg_pos={0 0}, upload=25760]: Done uploading piece. err: ecclient: upload failed (node:124d52DhtbtwipVpZuWdpFTQAttR18hy1K1BbokNRX4R1bLtzTi, address:104.36.82.53:4122): protocol: write tcp 159.223.143.39:58308->104.36.82.53:4122: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:52:32.241761782 (27h0m48.022695158s) [noise=true , address=104.36.82.14:4248    , piece_id=MZK5HPSINMTZVFBVAKY72SP6ILQ2YFHXHJMDWYC7DYZ3PWB4WQYA, seg_pos={0 0}, upload=25760]: Done uploading piece. err: ecclient: upload failed (node:1u6ir2PLqDa9xGm64cuqq1RqY4qeBa1fqCo9VghspBf54XEXqn, address:104.36.82.14:4248): protocol: write tcp 159.223.143.39:35248->104.36.82.14:4248: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:52:32.241814308 (27h0m48.022747686s) [noise=true , address=104.36.82.5:4855     , piece_id=DWG6KJOO2GWFOLAGKTD7YLERA2RAVYK54K727QDCEO6PTS3AHDQA, seg_pos={0 0}, upload=25760]: Done uploading piece. err: ecclient: upload failed (node:1KDNuE7YY8smhDjotushQvY3y9NtLxigbJELHD4Edxy8Qi4BWh, address:104.36.82.5:4855): protocol: write tcp 159.223.143.39:58916->104.36.82.5:4855: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:52:32.241825107 (27h0m48.022758489s) [noise=true , address=104.36.82.9:4038     , piece_id=TKAYQNRHYN7FP7P34IHLG4WJJWHNY75NCWGKD46PK3NXQN3BKK2A, seg_pos={0 0}, upload=25760]: Done uploading piece. err: ecclient: upload failed (node:12VTVM9YuevPA4vsWExjpyW5B9pqyEKKruUwnj1jNhE3j1ScCBg, address:104.36.82.9:4038): protocol: write tcp 159.223.143.39:34130->104.36.82.9:4038: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:52:32.241893130 (27h0m48.022826517s) [seg_pos={0 0}, upload=25760]: Done waiting for segment. successful: 1 optimal: 14 errs: <nil>
Jul 28 00:52:32.270675916 (27h0m48.051609285s) [upload=25760]: Done uploading segments...
Jul 28 00:52:32.270705520 (27h0m48.051638901s) [upload=25760]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 1)
Duration: 1m0.033359311s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
7 out of 60 is failed (threshold=3) [.....X........X.............XX...........X...X...X..........]
duration: 
537.784435ms
FAIL (2025-07-28 00:52:06 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
0}, upload=25651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 159.223.143.39:39890->104.36.82.7:4909: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:44:18.975782256 (26h52m34.756715638s) [seg_pos={0 0}, upload=25651]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 28 00:44:18.975871874 (26h52m34.756805277s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 28 00:44:18.975878645 (26h52m34.756812034s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 28 00:44:19.019856339 (26h52m34.800789708s) [upload=25651]: Done uploading segments...
Jul 28 00:44:19.019879905 (26h52m34.800813283s) [upload=25651]: Upload finished. err: <nil>
Jul 28 00:45:02.257192553 (26h53m18.03812594s) [upload=25659]: Starting upload
Jul 28 00:45:02.257306836 (26h53m18.038240216s) [upload=25659]: Done starting upload
Jul 28 00:45:02.257434360 (26h53m18.038367739s) [upload=25659]: Uploading segments...
Jul 28 00:45:02.257858888 (26h53m18.038792251s) [upload=25659]: Got next segment. Inline: false
Jul 28 00:45:02.257951856 (26h53m18.038885235s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 28 00:45:02.257959915 (26h53m18.038893293s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 28 00:45:02.293048419 (26h53m18.073981806s) [seg_pos={0 0}, upload=25659]: Begin upload segment...
Jul 28 00:45:02.293125122 (26h53m18.074058491s) [seg_pos={0 0}, upload=25659]: Done begin upload segment.
Jul 28 00:45:02.293171689 (26h53m18.074105073s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293724212 (26h53m18.074657618s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293744577 (26h53m18.074677964s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294292781 (26h53m18.075226212s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294343706 (26h53m18.075277094s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294756878 (26h53m18.075690269s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294891918 (26h53m18.075825306s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295221121 (26h53m18.076154513s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295558583 (26h53m18.076491975s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295604310 (26h53m18.0765377s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295991730 (26h53m18.07692512s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.296088221 (26h53m18.077021615s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.301695563 (26h53m18.082628952s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302203775 (26h53m18.083137164s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302629001 (26h53m18.083562389s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302995157 (26h53m18.083928524s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.418208178 (26h53m18.19914155s) [upload=25659]: Next returned nil segment
Jul 28 00:45:02.418226832 (26h53m18.19916021s) [upload=25659]: Waiting for error group managing segments...
Jul 28 00:45:02.424117363 (26h53m18.205050731s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.427234519 (26h53m18.208167896s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.430114806 (26h53m18.211048185s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.435624120 (26h53m18.216557505s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436119655 (26h53m18.217053027s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436405218 (26h53m18.217338582s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436887494 (26h53m18.217820863s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436998726 (26h53m18.217932114s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437527079 (26h53m18.218460459s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437625274 (26h53m18.218558642s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439469614 (26h53m18.220402995s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439557143 (26h53m18.220490506s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:32.221331076 (26h53m48.002264449s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: write tcp 159.223.143.39:33378->104.36.82.16:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221411781 (26h53m48.002345162s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221457184 (26h53m48.002390547s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: rpc: dial tcp 104.36.82.13:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221473904 (26h53m48.002407294s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:49546->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221516309 (26h53m48.002449671s) [seg_pos={0 0}, upload=25659]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul 28 00:45:32.251981717 (26h53m48.032915086s) [upload=25659]: Done uploading segments...
Jul 28 00:45:32.252005327 (26h53m48.032938704s) [upload=25659]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 537.817002ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip: 
7 out of 60 is failed (threshold=3) [.........X...XX.......X...........X...................X....X]
duration: 
30.033542s
FAIL (2025-07-28 00:51:55 UTC) 
orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 30.033620612s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
7 out of 60 is failed (threshold=3) [......X........X.............XX...........X...X...X.........]
duration: 
526.449487ms
FAIL (2025-07-28 00:51:49 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
0}, upload=25651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 159.223.143.39:39890->104.36.82.7:4909: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:44:18.975782256 (26h52m34.756715638s) [seg_pos={0 0}, upload=25651]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 28 00:44:18.975871874 (26h52m34.756805277s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 28 00:44:18.975878645 (26h52m34.756812034s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 28 00:44:19.019856339 (26h52m34.800789708s) [upload=25651]: Done uploading segments...
Jul 28 00:44:19.019879905 (26h52m34.800813283s) [upload=25651]: Upload finished. err: <nil>
Jul 28 00:45:02.257192553 (26h53m18.03812594s) [upload=25659]: Starting upload
Jul 28 00:45:02.257306836 (26h53m18.038240216s) [upload=25659]: Done starting upload
Jul 28 00:45:02.257434360 (26h53m18.038367739s) [upload=25659]: Uploading segments...
Jul 28 00:45:02.257858888 (26h53m18.038792251s) [upload=25659]: Got next segment. Inline: false
Jul 28 00:45:02.257951856 (26h53m18.038885235s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 28 00:45:02.257959915 (26h53m18.038893293s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 28 00:45:02.293048419 (26h53m18.073981806s) [seg_pos={0 0}, upload=25659]: Begin upload segment...
Jul 28 00:45:02.293125122 (26h53m18.074058491s) [seg_pos={0 0}, upload=25659]: Done begin upload segment.
Jul 28 00:45:02.293171689 (26h53m18.074105073s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293724212 (26h53m18.074657618s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293744577 (26h53m18.074677964s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294292781 (26h53m18.075226212s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294343706 (26h53m18.075277094s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294756878 (26h53m18.075690269s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294891918 (26h53m18.075825306s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295221121 (26h53m18.076154513s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295558583 (26h53m18.076491975s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295604310 (26h53m18.0765377s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295991730 (26h53m18.07692512s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.296088221 (26h53m18.077021615s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.301695563 (26h53m18.082628952s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302203775 (26h53m18.083137164s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302629001 (26h53m18.083562389s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302995157 (26h53m18.083928524s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.418208178 (26h53m18.19914155s) [upload=25659]: Next returned nil segment
Jul 28 00:45:02.418226832 (26h53m18.19916021s) [upload=25659]: Waiting for error group managing segments...
Jul 28 00:45:02.424117363 (26h53m18.205050731s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.427234519 (26h53m18.208167896s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.430114806 (26h53m18.211048185s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.435624120 (26h53m18.216557505s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436119655 (26h53m18.217053027s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436405218 (26h53m18.217338582s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436887494 (26h53m18.217820863s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436998726 (26h53m18.217932114s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437527079 (26h53m18.218460459s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437625274 (26h53m18.218558642s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439469614 (26h53m18.220402995s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439557143 (26h53m18.220490506s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:32.221331076 (26h53m48.002264449s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: write tcp 159.223.143.39:33378->104.36.82.16:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221411781 (26h53m48.002345162s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221457184 (26h53m48.002390547s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: rpc: dial tcp 104.36.82.13:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221473904 (26h53m48.002407294s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:49546->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221516309 (26h53m48.002449671s) [seg_pos={0 0}, upload=25659]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul 28 00:45:32.251981717 (26h53m48.032915086s) [upload=25659]: Done uploading segments...
Jul 28 00:45:32.252005327 (26h53m48.032938704s) [upload=25659]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 526.473698ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)
antiblip: 
45 out of 60 is failed (threshold=3) [..............XXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration: 
1m0.031137154s
FAIL (2025-07-28 00:51:32 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 2)
======LOGS======
05.233:4644  , piece_id=6IXCQRC4SEDQA7MQB34CFYZVMTX3SNA4TK55MGZQNUYFBDMXSIEA, seg_pos={0 0}, upload=25744]: Uploading piece...
Jul 28 00:50:32.210459133 (26h58m47.991392523s) [noise=true , address=79.127.201.215:4358  , piece_id=2KIJCG5R4RXSFN4HMP7DFEOBMLAMSL6U2DO7OYHCZYG5VWCOMCTQ, seg_pos={0 0}, upload=25744]: Uploading piece...
Jul 28 00:50:32.210981444 (26h58m47.991914827s) [noise=true , address=104.36.82.16:4298    , piece_id=L6QZO3ZBEHRC7LVEXINEEZVBPVMD4LOSXKGXMXEHMJYON7Z4WO3Q, seg_pos={0 0}, upload=25744]: Uploading piece...
Jul 28 00:50:32.212167520 (26h58m47.993100904s) [noise=true , address=104.36.82.55:4207    , piece_id=VX2SXBNMEJTLXXGA5LYZPQD6C7NVRYWXTDAK6OSNM4WZBDB7B72A, seg_pos={0 0}, upload=25744]: Uploading piece...
Jul 28 00:50:32.212859109 (26h58m47.993792489s) [noise=true , address=104.36.82.12:4166    , piece_id=W7UKAA5ELBQ7UPGEH6ARHXMGS2V7TXNV5KAI2RC2UQVOXD3HMQLQ, seg_pos={0 0}, upload=25744]: Uploading piece...
Jul 28 00:50:32.215422638 (26h58m47.996356027s) [noise=true , address=104.36.82.41:4779    , piece_id=Z7KHZCEMXOQIQHCQQTNSQRBWW5BPVJSMW44OJPJGZFAMUW7V4S7Q, seg_pos={0 0}, upload=25744]: Uploading piece...
Jul 28 00:50:32.216034703 (26h58m47.996968091s) [noise=true , address=104.36.82.36:4630    , piece_id=UFQU75P3AVM2VGLZNRVC45BLAT7EIXONGQIDSPCCQ57BJ4TUB47Q, seg_pos={0 0}, upload=25744]: Uploading piece...
Jul 28 00:50:32.217204721 (26h58m47.998138105s) [noise=true , address=104.36.82.54:4185    , piece_id=KNZATLL6JCGJBHQ2HXYMVGEVGHB6W4SYMC6TLXAE4IQCCUW4SFGA, seg_pos={0 0}, upload=25744]: Uploading piece...
Jul 28 00:50:32.217679814 (26h58m47.998613214s) [noise=true , address=104.36.82.46:4513    , piece_id=7BT6P5PLOHOK25GJ3VRNQZ3PIV7XIE3YCFQUPZOQOUFPD6YHHVEA, seg_pos={0 0}, upload=25744]: Uploading piece...
Jul 28 00:50:32.291120617 (26h58m48.072054001s) [upload=25744]: Next returned nil segment
Jul 28 00:50:32.291141617 (26h58m48.072075005s) [upload=25744]: Waiting for error group managing segments...
Jul 28 00:50:32.298287514 (26h58m48.079220883s) [noise=true , address=79.127.219.38:4002   , piece_id=VNUPVP462U6IKCYATB2JHYQ4I5CX7D5JGAYSZZ25A2KECYSEGQFQ, seg_pos={0 0}, upload=25744]: Done uploading piece. err: <nil>
Jul 28 00:50:32.574361999 (26h58m48.355295368s) [noise=true , address=79.127.205.233:4644  , piece_id=6IXCQRC4SEDQA7MQB34CFYZVMTX3SNA4TK55MGZQNUYFBDMXSIEA, seg_pos={0 0}, upload=25744]: Done uploading piece. err: ecclient: upload failed (node:12Du3rEYZrFAc7ZUHUvfS3DrDgs9X2zBmhQeTvEKJAjhQ3pqVkL, address:79.127.205.233:4644): protocol: storage node overloaded, request limit: 2000
Jul 28 00:50:42.287041210 (26h58m58.067974586s) [noise=true , address=79.127.201.215:4358  , piece_id=2KIJCG5R4RXSFN4HMP7DFEOBMLAMSL6U2DO7OYHCZYG5VWCOMCTQ, seg_pos={0 0}, upload=25744]: Done uploading piece. err: <nil>
Jul 28 00:51:32.126973977 (26h59m47.907907349s) [noise=true , address=104.36.82.46:4513    , piece_id=7BT6P5PLOHOK25GJ3VRNQZ3PIV7XIE3YCFQUPZOQOUFPD6YHHVEA, seg_pos={0 0}, upload=25744]: Done uploading piece. err: ecclient: upload failed (node:12pyyW8Gdqg6j9YGMtCUBhFnoNXryvPMUy19x815dp2HV3Xqr8h, address:104.36.82.46:4513): protocol: write tcp 159.223.143.39:51604->104.36.82.46:4513: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:51:32.127147923 (26h59m47.908081295s) [noise=true , address=104.36.82.14:4251    , piece_id=R32QJCJTDGYQZPN7UBLZCGK62EKLST4LYKMYF5YKIAC6YQL3EWXQ, seg_pos={0 0}, upload=25744]: Done uploading piece. err: ecclient: upload failed (node:12btf6Z38aTMx7MaBxhUyZtFqUPgeMtP2aU8Y6FbV8zwu57zArd, address:104.36.82.14:4251): protocol: write tcp 159.223.143.39:40514->104.36.82.14:4251: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:51:32.127909398 (26h59m47.908842777s) [noise=true , address=104.36.82.56:4261    , piece_id=DQMGFK6CZJYU6FOV3MLNICL6CGJ6AGNWBPQ3WDX2RH7CLDX5M4HA, seg_pos={0 0}, upload=25744]: Done uploading piece. err: ecclient: upload failed (node:12YNKaVEsXzHe6XGQuv6wR1qqbv6dbzg1xT5Aqkew2ZV3z3tMMf, address:104.36.82.56:4261): protocol: rpc: dial tcp 104.36.82.56:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:51:32.128076760 (26h59m47.909010122s) [noise=true , address=104.36.82.42:4806    , piece_id=Q4KNE6TJTKR573WZCLXN4TOXGZCBLS54UFF7W3PY5L3AXTYVYL7Q, seg_pos={0 0}, upload=25744]: Done uploading piece. err: ecclient: upload failed (node:1tWnxuTSo4yHAJUzvVFTmwPSNb8cBJJJ17BnKd5Z3HMYf66vSC, address:104.36.82.42:4806): protocol: rpc: dial tcp 104.36.82.42:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:51:32.128360832 (26h59m47.909294411s) [noise=true , address=104.36.82.12:4166    , piece_id=W7UKAA5ELBQ7UPGEH6ARHXMGS2V7TXNV5KAI2RC2UQVOXD3HMQLQ, seg_pos={0 0}, upload=25744]: Done uploading piece. err: ecclient: upload failed (node:12iCB7c4ZfpqafpD1PtpbE6BfU81wJW65LmrPVJXtD58yeuoR9t, address:104.36.82.12:4166): protocol: rpc: dial tcp 104.36.82.12:4166: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:51:32.128473124 (26h59m47.909406488s) [noise=true , address=104.36.82.36:4630    , piece_id=UFQU75P3AVM2VGLZNRVC45BLAT7EIXONGQIDSPCCQ57BJ4TUB47Q, seg_pos={0 0}, upload=25744]: Done uploading piece. err: ecclient: upload failed (node:12aETKgTzKw8fyYqbeDA2JzfxZnWn12xUPYK1eRt6TvBPPdRJDp, address:104.36.82.36:4630): protocol: rpc: dial tcp 104.36.82.36:4630: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:51:32.128487391 (26h59m47.909420756s) [noise=true , address=104.36.82.52:4093    , piece_id=2WBOBMEXOPDBR7KJ6SZKGL2XWF36G7JTEZ24T7XP7INGHAS3KU4A, seg_pos={0 0}, upload=25744]: Done uploading piece. err: ecclient: upload failed (node:12APDQdo9gz4mqC7TxxZE6zbf92LLP26iUYfpcpri4LyQn2ic4E, address:104.36.82.52:4093): protocol: write tcp 159.223.143.39:60488->104.36.82.52:4093: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:51:32.128556560 (26h59m47.90948994s) [noise=true , address=104.36.82.53:4141    , piece_id=JO2RGZFG6U553UPRUBWC34NXGNOLNCW5C5GUPXLSP5BMEQXKW7LQ, seg_pos={0 0}, upload=25744]: Done uploading piece. err: ecclient: upload failed (node:15wUF135h4QsyvF7Cr3eb6rxtx8KPz1LHbgRjQYwiF5xpmE2nn, address:104.36.82.53:4141): protocol: rpc: dial tcp 104.36.82.53:4141: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:51:32.128610240 (26h59m47.909543613s) [noise=true , address=104.36.82.54:4185    , piece_id=KNZATLL6JCGJBHQ2HXYMVGEVGHB6W4SYMC6TLXAE4IQCCUW4SFGA, seg_pos={0 0}, upload=25744]: Done uploading piece. err: ecclient: upload failed (node:1cLdyyVLrp1yBmgsfBrfYc6N2RFbBichYoBbo2F7EZfaBXtkfk, address:104.36.82.54:4185): protocol: rpc: dial tcp 104.36.82.54:4185: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:51:32.128622921 (26h59m47.909556288s) [noise=true , address=104.36.82.46:4488    , piece_id=CHX7UTIWQHSFMU2Q2O7XC7MADPT7OYKOGWJSOK6X6GQ2VTNPQXJQ, seg_pos={0 0}, upload=25744]: Done uploading piece. err: ecclient: upload failed (node:1MU3fdv9cTkq4pUZ3h989STAG1wxrbhFgzmr6tcrfumZKHP9BZ, address:104.36.82.46:4488): protocol: rpc: dial tcp 104.36.82.46:4488: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:51:32.128673268 (26h59m47.909606643s) [noise=true , address=104.36.82.53:4122    , piece_id=YHZDSNGVRKQKX5WUWSMNVYSK6JFY7UMQYV3MWG4TFECVJG75MTKA, seg_pos={0 0}, upload=25744]: Done uploading piece. err: ecclient: upload failed (node:124d52DhtbtwipVpZuWdpFTQAttR18hy1K1BbokNRX4R1bLtzTi, address:104.36.82.53:4122): protocol: write tcp 159.223.143.39:54644->104.36.82.53:4122: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:51:32.128687736 (26h59m47.909621101s) [noise=true , address=104.36.82.16:4298    , piece_id=L6QZO3ZBEHRC7LVEXINEEZVBPVMD4LOSXKGXMXEHMJYON7Z4WO3Q, seg_pos={0 0}, upload=25744]: Done uploading piece. err: ecclient: upload failed (node:1sDggu72xPTG9o2K8sbPHbYfwXarrvqP98YySo7ZSEZ2ddYFEz, address:104.36.82.16:4298): protocol: rpc: dial tcp 104.36.82.16:4298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:51:32.128733357 (26h59m47.909666727s) [noise=true , address=104.36.82.11:4143    , piece_id=UHOOV7TKTSM3BGWVHBBFCJHXHAUCFJQPHJ5RKRIBIWQALZHA2HBQ, seg_pos={0 0}, upload=25744]: Done uploading piece. err: ecclient: upload failed (node:12ipThd6PwZUpCXy3ML4qqMd9dmRLmxKjVNCNoT5wCg1szdWp8U, address:104.36.82.11:4143): protocol: write tcp 159.223.143.39:34690->104.36.82.11:4143: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:51:32.128745867 (26h59m47.909679247s) [noise=true , address=104.36.82.41:4779    , piece_id=Z7KHZCEMXOQIQHCQQTNSQRBWW5BPVJSMW44OJPJGZFAMUW7V4S7Q, seg_pos={0 0}, upload=25744]: Done uploading piece. err: ecclient: upload failed (node:1eRovJAp3AyUHP4qkuDvSSGYb9TG6U4tSNzniGJBeYLhkmiUVy, address:104.36.82.41:4779): protocol: rpc: dial tcp 104.36.82.41:4779: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:51:32.128792846 (26h59m47.909726212s) [noise=true , address=104.36.82.55:4207    , piece_id=VX2SXBNMEJTLXXGA5LYZPQD6C7NVRYWXTDAK6OSNM4WZBDB7B72A, seg_pos={0 0}, upload=25744]: Done uploading piece. err: ecclient: upload failed (node:121hXqdHirzNQq5H1ynCC3bboxCyYpeRfBbZR4zPQCPiW2ebbey, address:104.36.82.55:4207): protocol: rpc: dial tcp 104.36.82.55:4207: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:51:32.128827350 (26h59m47.909760728s) [seg_pos={0 0}, upload=25744]: Done waiting for segment. successful: 2 optimal: 14 errs: <nil>
Jul 28 00:51:32.156690152 (26h59m47.937623522s) [upload=25744]: Done uploading segments...
Jul 28 00:51:32.156714330 (26h59m47.937647695s) [upload=25744]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 2)
Duration: 1m0.031155833s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
7 out of 60 is failed (threshold=3) [.......X........X.............XX...........X...X...X........]
duration: 
421.659247ms
FAIL (2025-07-28 00:51:16 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
0}, upload=25651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 159.223.143.39:39890->104.36.82.7:4909: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:44:18.975782256 (26h52m34.756715638s) [seg_pos={0 0}, upload=25651]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 28 00:44:18.975871874 (26h52m34.756805277s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 28 00:44:18.975878645 (26h52m34.756812034s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 28 00:44:19.019856339 (26h52m34.800789708s) [upload=25651]: Done uploading segments...
Jul 28 00:44:19.019879905 (26h52m34.800813283s) [upload=25651]: Upload finished. err: <nil>
Jul 28 00:45:02.257192553 (26h53m18.03812594s) [upload=25659]: Starting upload
Jul 28 00:45:02.257306836 (26h53m18.038240216s) [upload=25659]: Done starting upload
Jul 28 00:45:02.257434360 (26h53m18.038367739s) [upload=25659]: Uploading segments...
Jul 28 00:45:02.257858888 (26h53m18.038792251s) [upload=25659]: Got next segment. Inline: false
Jul 28 00:45:02.257951856 (26h53m18.038885235s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 28 00:45:02.257959915 (26h53m18.038893293s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 28 00:45:02.293048419 (26h53m18.073981806s) [seg_pos={0 0}, upload=25659]: Begin upload segment...
Jul 28 00:45:02.293125122 (26h53m18.074058491s) [seg_pos={0 0}, upload=25659]: Done begin upload segment.
Jul 28 00:45:02.293171689 (26h53m18.074105073s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293724212 (26h53m18.074657618s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293744577 (26h53m18.074677964s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294292781 (26h53m18.075226212s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294343706 (26h53m18.075277094s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294756878 (26h53m18.075690269s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294891918 (26h53m18.075825306s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295221121 (26h53m18.076154513s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295558583 (26h53m18.076491975s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295604310 (26h53m18.0765377s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295991730 (26h53m18.07692512s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.296088221 (26h53m18.077021615s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.301695563 (26h53m18.082628952s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302203775 (26h53m18.083137164s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302629001 (26h53m18.083562389s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302995157 (26h53m18.083928524s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.418208178 (26h53m18.19914155s) [upload=25659]: Next returned nil segment
Jul 28 00:45:02.418226832 (26h53m18.19916021s) [upload=25659]: Waiting for error group managing segments...
Jul 28 00:45:02.424117363 (26h53m18.205050731s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.427234519 (26h53m18.208167896s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.430114806 (26h53m18.211048185s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.435624120 (26h53m18.216557505s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436119655 (26h53m18.217053027s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436405218 (26h53m18.217338582s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436887494 (26h53m18.217820863s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436998726 (26h53m18.217932114s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437527079 (26h53m18.218460459s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437625274 (26h53m18.218558642s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439469614 (26h53m18.220402995s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439557143 (26h53m18.220490506s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:32.221331076 (26h53m48.002264449s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: write tcp 159.223.143.39:33378->104.36.82.16:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221411781 (26h53m48.002345162s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221457184 (26h53m48.002390547s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: rpc: dial tcp 104.36.82.13:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221473904 (26h53m48.002407294s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:49546->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221516309 (26h53m48.002449671s) [seg_pos={0 0}, upload=25659]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul 28 00:45:32.251981717 (26h53m48.032915086s) [upload=25659]: Done uploading segments...
Jul 28 00:45:32.252005327 (26h53m48.032938704s) [upload=25659]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 421.687583ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)
antiblip: 
44 out of 60 is failed (threshold=3) [...............XXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration: 
1m0.032836538s
FAIL (2025-07-28 00:50:32 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 1)
======LOGS======
6h57m47.874429428s) [noise=true , address=104.36.82.11:4138    , piece_id=GBG2Y257WSXK6UJ33NNP3JII7VYIHWR2BIAQZ7AEMJOOJ3Z2ULLA, seg_pos={0 0}, upload=25729]: Uploading piece...
Jul 28 00:49:32.093913552 (26h57m47.874846937s) [noise=true , address=104.36.82.21:4501    , piece_id=IOP3WG2QDNX56MWIARHLDBETAEJAPK2322VFHHKNJG2QT25V5CBQ, seg_pos={0 0}, upload=25729]: Uploading piece...
Jul 28 00:49:32.094394373 (26h57m47.875327759s) [noise=true , address=104.36.82.56:4247    , piece_id=PJBS6XMAIQ73KADL6MTBIULJEQB6IYTUMZPBNWTBSNIO2TD6WD7Q, seg_pos={0 0}, upload=25729]: Uploading piece...
Jul 28 00:49:32.094794940 (26h57m47.875728327s) [noise=true , address=104.36.82.13:4206    , piece_id=JEHCSGB25YF5FXJ3YZTDZU2SVQ42JY57I3JERBN3IY7E6ZINY7KQ, seg_pos={0 0}, upload=25729]: Uploading piece...
Jul 28 00:49:32.095309644 (26h57m47.876243028s) [noise=true , address=104.36.82.42:4810    , piece_id=Q457XU7Z5WI6DH6DPYRFE3CAJB6C4NCV2KBLBFTI3ODD2TO3JESA, seg_pos={0 0}, upload=25729]: Uploading piece...
Jul 28 00:49:32.095767961 (26h57m47.876701347s) [noise=true , address=104.36.82.7:4930     , piece_id=6JUNE32FAIURQOA3LRM4MDRU6RSSU6EJAAPQZ37BGV64OZEFID4A, seg_pos={0 0}, upload=25729]: Uploading piece...
Jul 28 00:49:32.096218339 (26h57m47.877151725s) [noise=true , address=104.36.82.52:4111    , piece_id=QVUSLEPJL62RAXAYJESEFFR3NM4AFOWBYKEJK2MPTEJ2NIFDDN3Q, seg_pos={0 0}, upload=25729]: Uploading piece...
Jul 28 00:49:32.096640818 (26h57m47.877574202s) [noise=true , address=104.36.82.55:4212    , piece_id=7RBUWSB4WYS5TZIAAA7NRSWSGWNZSQPCQLP5VZXFZJOINOWZIPKQ, seg_pos={0 0}, upload=25729]: Uploading piece...
Jul 28 00:49:32.305760666 (26h57m48.086694033s) [upload=25729]: Next returned nil segment
Jul 28 00:49:32.305777689 (26h57m48.086711064s) [upload=25729]: Waiting for error group managing segments...
Jul 28 00:49:38.362365404 (26h57m54.143298772s) [noise=true , address=79.127.201.215:4344  , piece_id=CDVKCM2IP376NKJ3DNJSH26KLWTGTESNIIAA7X52C2DV57JDPI2A, seg_pos={0 0}, upload=25729]: Done uploading piece. err: <nil>
Jul 28 00:50:32.027254432 (26h58m47.808187813s) [noise=true , address=104.36.82.46:4513    , piece_id=YPQD7IW4PTWX4Y4TD6XJRLEN3HBWCFD5AAGSYDUE44MSEPFXKCDQ, seg_pos={0 0}, upload=25729]: Done uploading piece. err: ecclient: upload failed (node:12pyyW8Gdqg6j9YGMtCUBhFnoNXryvPMUy19x815dp2HV3Xqr8h, address:104.36.82.46:4513): protocol: rpc: dial tcp 104.36.82.46:4513: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:50:32.027359118 (26h58m47.808292578s) [noise=true , address=104.36.82.11:4138    , piece_id=GBG2Y257WSXK6UJ33NNP3JII7VYIHWR2BIAQZ7AEMJOOJ3Z2ULLA, seg_pos={0 0}, upload=25729]: Done uploading piece. err: ecclient: upload failed (node:1H28XSTvFGa3XhavfhythHWs95cqjazu2RuqpsjNCmdSFvRLoh, address:104.36.82.11:4138): protocol: write tcp 159.223.143.39:58070->104.36.82.11:4138: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:50:32.027430008 (26h58m47.80836339s) [noise=true , address=104.36.82.13:4206    , piece_id=JEHCSGB25YF5FXJ3YZTDZU2SVQ42JY57I3JERBN3IY7E6ZINY7KQ, seg_pos={0 0}, upload=25729]: Done uploading piece. err: ecclient: upload failed (node:1ih3m2bU4PfDMX5zfXoGw3t2eyY1u6wVnpqczo2sz7aTq99R9T, address:104.36.82.13:4206): protocol: rpc: dial tcp 104.36.82.13:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:50:32.027503381 (26h58m47.808436749s) [noise=true , address=104.36.82.3:4794     , piece_id=FZU5HJM3XA3GP7XM5T3IZZV55P2VDMH3GJGUNRY47B7A6JN24S6A, seg_pos={0 0}, upload=25729]: Done uploading piece. err: ecclient: upload failed (node:12ctLLD5skk4t5i3BFYgoTmuZ2aqsX23wDGDMnKriQ7U2KXyPAo, address:104.36.82.3:4794): protocol: rpc: dial tcp 104.36.82.3:4794: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:50:32.027556066 (26h58m47.808489433s) [noise=true , address=104.36.82.12:4175    , piece_id=D6ISJOR5E33RSDTFCLV5POD2TCIGGUMWAVXAAB7RS4VP2MK53GFA, seg_pos={0 0}, upload=25729]: Done uploading piece. err: ecclient: upload failed (node:12JEJYMUS93YZ9Vxx6LT6psJmt2gn569s7dV7mBJtLsjh2e9zKE, address:104.36.82.12:4175): protocol: rpc: dial tcp 104.36.82.12:4175: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:50:32.027615470 (26h58m47.808548837s) [noise=true , address=104.36.82.20:4459    , piece_id=KMTHW6OCEMSA6RI5IPBTUIX2VGG37K7UMNVT77DTTGO5L5LCPQCQ, seg_pos={0 0}, upload=25729]: Done uploading piece. err: ecclient: upload failed (node:1hpq5G467oUtEaE6Cj842pHCCZ1V7EraP5EDdGdWMxz4M3Q3pA, address:104.36.82.20:4459): protocol: rpc: dial tcp 104.36.82.20:4459: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:50:32.027626911 (26h58m47.808560302s) [noise=true , address=104.36.82.36:4646    , piece_id=MKWYZQAZ3A3TXENNAIGYCFPZUDFFBXT4FVUKZPWYFDGRLRIZJ4NA, seg_pos={0 0}, upload=25729]: Done uploading piece. err: ecclient: upload failed (node:1zA8TqySnmGc9bVQnNNMSZEPbv9Gf1Wa1BwYrKjc2ZGgtyEmrh, address:104.36.82.36:4646): protocol: rpc: dial tcp 104.36.82.36:4646: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:50:32.027659353 (26h58m47.80859272s) [noise=true , address=104.36.82.47:4533    , piece_id=BXVHUM52NY6B7XOSPOSCQQ75UEYLPXXLA4ZO23OUKOA5SBVSPRCQ, seg_pos={0 0}, upload=25729]: Done uploading piece. err: ecclient: upload failed (node:1JUUdAnaUPAE8tkzPvVCokFbZMwmhqZvwKHXReK9VU2QeDqavR, address:104.36.82.47:4533): protocol: rpc: dial tcp 104.36.82.47:4533: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:50:32.027705300 (26h58m47.808638683s) [noise=true , address=104.36.82.21:4501    , piece_id=IOP3WG2QDNX56MWIARHLDBETAEJAPK2322VFHHKNJG2QT25V5CBQ, seg_pos={0 0}, upload=25729]: Done uploading piece. err: ecclient: upload failed (node:1e1R3gjh5CYJdDqck7xcR6A19Hb443FWbX8X2AQikxt5f4AVvU, address:104.36.82.21:4501): protocol: rpc: dial tcp 104.36.82.21:4501: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:50:32.027720735 (26h58m47.808654113s) [noise=true , address=104.36.82.42:4810    , piece_id=Q457XU7Z5WI6DH6DPYRFE3CAJB6C4NCV2KBLBFTI3ODD2TO3JESA, seg_pos={0 0}, upload=25729]: Done uploading piece. err: ecclient: upload failed (node:1CGnNCsXi1RAnTx9P28561Q1wK6LBqpBzqyuowxaAZJETQRz2w, address:104.36.82.42:4810): protocol: rpc: dial tcp 104.36.82.42:4810: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:50:32.027772753 (26h58m47.808706134s) [noise=true , address=104.36.82.56:4247    , piece_id=PJBS6XMAIQ73KADL6MTBIULJEQB6IYTUMZPBNWTBSNIO2TD6WD7Q, seg_pos={0 0}, upload=25729]: Done uploading piece. err: ecclient: upload failed (node:12PzKreuPT9X594ZTTYbXanNdaFVVJsnKL99fumjXymnRa9jeEq, address:104.36.82.56:4247): protocol: rpc: dial tcp 104.36.82.56:4247: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:50:32.027786346 (26h58m47.808719713s) [noise=true , address=104.36.82.55:4212    , piece_id=7RBUWSB4WYS5TZIAAA7NRSWSGWNZSQPCQLP5VZXFZJOINOWZIPKQ, seg_pos={0 0}, upload=25729]: Done uploading piece. err: ecclient: upload failed (node:1EPo99poctfFTjokBfrawXtUeyraudrXQhXFeyQGe72mjD789y, address:104.36.82.55:4212): protocol: rpc: dial tcp 104.36.82.55:4212: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:50:32.027839711 (26h58m47.80877308s) [noise=true , address=104.36.82.52:4111    , piece_id=QVUSLEPJL62RAXAYJESEFFR3NM4AFOWBYKEJK2MPTEJ2NIFDDN3Q, seg_pos={0 0}, upload=25729]: Done uploading piece. err: ecclient: upload failed (node:12krqZcFoBPfLFbLGQUAjPMzHJ4HKqsKPRg6Xap7nuShaEKBumH, address:104.36.82.52:4111): protocol: rpc: dial tcp 104.36.82.52:4111: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:50:32.027902804 (26h58m47.808836183s) [noise=true , address=104.36.82.2:4742     , piece_id=ATO4KA6SHBUYUSYLOWNANS55G4FSMTGZBQ4IUHK3IESKTSZQYI4Q, seg_pos={0 0}, upload=25729]: Done uploading piece. err: ecclient: upload failed (node:1WsGo4te3PENFo9YRioYFrhkeRQvztnLVZJV4i38TV3TjKqBFT, address:104.36.82.2:4742): protocol: rpc: dial tcp 104.36.82.2:4742: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:50:32.027939728 (26h58m47.808873091s) [noise=true , address=104.36.82.47:4536    , piece_id=D34A3ZOVJAVZYADJKJNTC4LDOIXHYDWPDNWKUO4AEJ3SV7DRFYMA, seg_pos={0 0}, upload=25729]: Done uploading piece. err: ecclient: upload failed (node:1WoVNAp5RfVviaxN4RWCUDEhxLKsdGB8bgE6gcUcgZKcM4Cz1s, address:104.36.82.47:4536): protocol: rpc: dial tcp 104.36.82.47:4536: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:50:32.027954782 (26h58m47.808888145s) [noise=true , address=104.36.82.54:4178    , piece_id=4KMXDSRZ2ER52CCNG2NTYXJKZTMMOZLDPCPUZWU34DFEEQSXSG5A, seg_pos={0 0}, upload=25729]: Done uploading piece. err: ecclient: upload failed (node:1jyRCtxSCu2bkU41Kgb1aeGppQLgX5iEo322Zoi1cWFkmGeYgr, address:104.36.82.54:4178): protocol: rpc: dial tcp 104.36.82.54:4178: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:50:32.027971194 (26h58m47.808904564s) [noise=true , address=104.36.82.7:4930     , piece_id=6JUNE32FAIURQOA3LRM4MDRU6RSSU6EJAAPQZ37BGV64OZEFID4A, seg_pos={0 0}, upload=25729]: Done uploading piece. err: ecclient: upload failed (node:1W215rQvfvgCZHym7547eECCCfSqEe2pe3R8jWRusjzNCp5Jac, address:104.36.82.7:4930): protocol: write tcp 159.223.143.39:41776->104.36.82.7:4930: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:50:32.028014066 (26h58m47.808947455s) [seg_pos={0 0}, upload=25729]: Done waiting for segment. successful: 1 optimal: 14 errs: <nil>
Jul 28 00:50:32.055936425 (26h58m47.836869806s) [upload=25729]: Done uploading segments...
Jul 28 00:50:32.055988855 (26h58m47.836922242s) [upload=25729]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 1)
Duration: 1m0.032874366s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip: 
6 out of 60 is failed (threshold=3) [..........X...XX.......X...........X...................X....]
duration: 
343.680103ms
FAIL (2025-07-28 00:50:27 UTC) 
orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 343.710621ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
7 out of 60 is failed (threshold=3) [........X........X.............XX...........X...X...X.......]
duration: 
445.751289ms
FAIL (2025-07-28 00:50:26 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
0}, upload=25651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 159.223.143.39:39890->104.36.82.7:4909: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:44:18.975782256 (26h52m34.756715638s) [seg_pos={0 0}, upload=25651]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 28 00:44:18.975871874 (26h52m34.756805277s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 28 00:44:18.975878645 (26h52m34.756812034s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 28 00:44:19.019856339 (26h52m34.800789708s) [upload=25651]: Done uploading segments...
Jul 28 00:44:19.019879905 (26h52m34.800813283s) [upload=25651]: Upload finished. err: <nil>
Jul 28 00:45:02.257192553 (26h53m18.03812594s) [upload=25659]: Starting upload
Jul 28 00:45:02.257306836 (26h53m18.038240216s) [upload=25659]: Done starting upload
Jul 28 00:45:02.257434360 (26h53m18.038367739s) [upload=25659]: Uploading segments...
Jul 28 00:45:02.257858888 (26h53m18.038792251s) [upload=25659]: Got next segment. Inline: false
Jul 28 00:45:02.257951856 (26h53m18.038885235s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 28 00:45:02.257959915 (26h53m18.038893293s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 28 00:45:02.293048419 (26h53m18.073981806s) [seg_pos={0 0}, upload=25659]: Begin upload segment...
Jul 28 00:45:02.293125122 (26h53m18.074058491s) [seg_pos={0 0}, upload=25659]: Done begin upload segment.
Jul 28 00:45:02.293171689 (26h53m18.074105073s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293724212 (26h53m18.074657618s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293744577 (26h53m18.074677964s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294292781 (26h53m18.075226212s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294343706 (26h53m18.075277094s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294756878 (26h53m18.075690269s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294891918 (26h53m18.075825306s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295221121 (26h53m18.076154513s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295558583 (26h53m18.076491975s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295604310 (26h53m18.0765377s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295991730 (26h53m18.07692512s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.296088221 (26h53m18.077021615s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.301695563 (26h53m18.082628952s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302203775 (26h53m18.083137164s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302629001 (26h53m18.083562389s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302995157 (26h53m18.083928524s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.418208178 (26h53m18.19914155s) [upload=25659]: Next returned nil segment
Jul 28 00:45:02.418226832 (26h53m18.19916021s) [upload=25659]: Waiting for error group managing segments...
Jul 28 00:45:02.424117363 (26h53m18.205050731s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.427234519 (26h53m18.208167896s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.430114806 (26h53m18.211048185s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.435624120 (26h53m18.216557505s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436119655 (26h53m18.217053027s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436405218 (26h53m18.217338582s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436887494 (26h53m18.217820863s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436998726 (26h53m18.217932114s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437527079 (26h53m18.218460459s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437625274 (26h53m18.218558642s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439469614 (26h53m18.220402995s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439557143 (26h53m18.220490506s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:32.221331076 (26h53m48.002264449s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: write tcp 159.223.143.39:33378->104.36.82.16:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221411781 (26h53m48.002345162s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221457184 (26h53m48.002390547s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: rpc: dial tcp 104.36.82.13:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221473904 (26h53m48.002407294s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:49546->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221516309 (26h53m48.002449671s) [seg_pos={0 0}, upload=25659]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul 28 00:45:32.251981717 (26h53m48.032915086s) [upload=25659]: Done uploading segments...
Jul 28 00:45:32.252005327 (26h53m48.032938704s) [upload=25659]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 445.783338ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip: 
6 out of 60 is failed (threshold=3) [...........X...XX.......X...........X...................X...]
duration: 
454.000241ms
FAIL (2025-07-28 00:49:39 UTC) 
orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 454.035522ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
7 out of 60 is failed (threshold=3) [.........X........X.............XX...........X...X...X......]
duration: 
417.742905ms
FAIL (2025-07-28 00:49:32 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
0}, upload=25651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 159.223.143.39:39890->104.36.82.7:4909: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:44:18.975782256 (26h52m34.756715638s) [seg_pos={0 0}, upload=25651]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 28 00:44:18.975871874 (26h52m34.756805277s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 28 00:44:18.975878645 (26h52m34.756812034s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 28 00:44:19.019856339 (26h52m34.800789708s) [upload=25651]: Done uploading segments...
Jul 28 00:44:19.019879905 (26h52m34.800813283s) [upload=25651]: Upload finished. err: <nil>
Jul 28 00:45:02.257192553 (26h53m18.03812594s) [upload=25659]: Starting upload
Jul 28 00:45:02.257306836 (26h53m18.038240216s) [upload=25659]: Done starting upload
Jul 28 00:45:02.257434360 (26h53m18.038367739s) [upload=25659]: Uploading segments...
Jul 28 00:45:02.257858888 (26h53m18.038792251s) [upload=25659]: Got next segment. Inline: false
Jul 28 00:45:02.257951856 (26h53m18.038885235s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 28 00:45:02.257959915 (26h53m18.038893293s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 28 00:45:02.293048419 (26h53m18.073981806s) [seg_pos={0 0}, upload=25659]: Begin upload segment...
Jul 28 00:45:02.293125122 (26h53m18.074058491s) [seg_pos={0 0}, upload=25659]: Done begin upload segment.
Jul 28 00:45:02.293171689 (26h53m18.074105073s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293724212 (26h53m18.074657618s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293744577 (26h53m18.074677964s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294292781 (26h53m18.075226212s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294343706 (26h53m18.075277094s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294756878 (26h53m18.075690269s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294891918 (26h53m18.075825306s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295221121 (26h53m18.076154513s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295558583 (26h53m18.076491975s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295604310 (26h53m18.0765377s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295991730 (26h53m18.07692512s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.296088221 (26h53m18.077021615s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.301695563 (26h53m18.082628952s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302203775 (26h53m18.083137164s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302629001 (26h53m18.083562389s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302995157 (26h53m18.083928524s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.418208178 (26h53m18.19914155s) [upload=25659]: Next returned nil segment
Jul 28 00:45:02.418226832 (26h53m18.19916021s) [upload=25659]: Waiting for error group managing segments...
Jul 28 00:45:02.424117363 (26h53m18.205050731s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.427234519 (26h53m18.208167896s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.430114806 (26h53m18.211048185s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.435624120 (26h53m18.216557505s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436119655 (26h53m18.217053027s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436405218 (26h53m18.217338582s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436887494 (26h53m18.217820863s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436998726 (26h53m18.217932114s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437527079 (26h53m18.218460459s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437625274 (26h53m18.218558642s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439469614 (26h53m18.220402995s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439557143 (26h53m18.220490506s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:32.221331076 (26h53m48.002264449s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: write tcp 159.223.143.39:33378->104.36.82.16:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221411781 (26h53m48.002345162s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221457184 (26h53m48.002390547s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: rpc: dial tcp 104.36.82.13:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221473904 (26h53m48.002407294s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:49546->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221516309 (26h53m48.002449671s) [seg_pos={0 0}, upload=25659]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul 28 00:45:32.251981717 (26h53m48.032915086s) [upload=25659]: Done uploading segments...
Jul 28 00:45:32.252005327 (26h53m48.032938704s) [upload=25659]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 417.774996ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)
antiblip: 
43 out of 60 is failed (threshold=3) [................XXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration: 
1m0.049714662s
FAIL (2025-07-28 00:49:31 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 3)
======LOGS======
ul 28 00:48:31.953146290 (26h56m47.734079677s) [noise=true , address=104.36.82.13:4210    , piece_id=FAEFTTKDGQLBP3RBFWUCFYDTMLDGIKXKX56ISSEOWZEXAXX5ZETA, seg_pos={0 0}, upload=25713]: Uploading piece...
Jul 28 00:48:31.953222823 (26h56m47.734156198s) [noise=true , address=104.36.82.10:4083    , piece_id=IOMZFD47YOMVLALTP3MC5EXXDB5K2Y3V37C5XIMSUIEFOANURW5A, seg_pos={0 0}, upload=25713]: Uploading piece...
Jul 28 00:48:31.953546584 (26h56m47.734479969s) [noise=true , address=104.36.82.14:4246    , piece_id=Z476KF7P7SEUR4O237EDCLUPSWGWQLK3J4PP6A6R3T2NITUC3OXA, seg_pos={0 0}, upload=25713]: Uploading piece...
Jul 28 00:48:31.954554398 (26h56m47.735487783s) [noise=true , address=79.127.205.246:4274  , piece_id=ANPPZXGVP2F5GAVNP42GB2SSGC2ZOWCCLGIQT5TYTOBR67JSH74Q, seg_pos={0 0}, upload=25713]: Uploading piece...
Jul 28 00:48:31.955475849 (26h56m47.736409234s) [noise=true , address=104.36.82.54:4180    , piece_id=SJCELFITI4HBVWK2N277FUUEZEGJBG6HN7VGU2Y6N7JXK5RNG6HA, seg_pos={0 0}, upload=25713]: Uploading piece...
Jul 28 00:48:31.956696401 (26h56m47.737629779s) [noise=true , address=104.36.82.22:4523    , piece_id=GUQYZN3KBIBMYA55CAZIYDRVT57QOMXQ2VQ55VLAQ6JQETDT7AIQ, seg_pos={0 0}, upload=25713]: Uploading piece...
Jul 28 00:48:31.957070099 (26h56m47.738003489s) [noise=true , address=104.36.82.47:4552    , piece_id=GED27X4HDTAYFKGSLSJPM54O2ZA5YJMVHADC26WTHISXRYVU5KFQ, seg_pos={0 0}, upload=25713]: Uploading piece...
Jul 28 00:48:31.957404479 (26h56m47.738337848s) [noise=true , address=104.36.82.40:4972    , piece_id=FLGCFR6Z5SIZRXCUTOKHRQ6S645VZZWSQYVFIWANCVXWEJ55AI4A, seg_pos={0 0}, upload=25713]: Uploading piece...
Jul 28 00:48:32.159766472 (26h56m47.940699847s) [upload=25713]: Next returned nil segment
Jul 28 00:48:32.159785691 (26h56m47.940719056s) [upload=25713]: Waiting for error group managing segments...
Jul 28 00:48:32.687870317 (26h56m48.468803685s) [noise=true , address=79.127.205.245:4247  , piece_id=OWM5QSDTTPBKXHCXLVFM7VNUQG6KJB467FYNOCHFSUEY27B73X7A, seg_pos={0 0}, upload=25713]: Done uploading piece. err: <nil>
Jul 28 00:48:32.784779967 (26h56m48.565713335s) [noise=true , address=79.127.205.246:4274  , piece_id=ANPPZXGVP2F5GAVNP42GB2SSGC2ZOWCCLGIQT5TYTOBR67JSH74Q, seg_pos={0 0}, upload=25713]: Done uploading piece. err: <nil>
Jul 28 00:48:37.097441220 (26h56m52.878374591s) [noise=true , address=79.127.201.215:4355  , piece_id=F44K2POGGLTT5FCZ763TQ7EOMANOCH7OYBVNQM2K6QFQU7E3PLTA, seg_pos={0 0}, upload=25713]: Done uploading piece. err: <nil>
Jul 28 00:49:31.910726877 (26h57m47.691660265s) [noise=true , address=104.36.82.54:4180    , piece_id=SJCELFITI4HBVWK2N277FUUEZEGJBG6HN7VGU2Y6N7JXK5RNG6HA, seg_pos={0 0}, upload=25713]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:104.36.82.54:4180): protocol: write tcp 159.223.143.39:48440->104.36.82.54:4180: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:49:31.910807284 (26h57m47.691740649s) [noise=true , address=104.36.82.21:4478    , piece_id=I4E2BSKQQBZ5HDWGEC4AS5Y54OMCGCHTKG2ANYWWAGAKD7TNB6FQ, seg_pos={0 0}, upload=25713]: Done uploading piece. err: ecclient: upload failed (node:17kFjjJyvHApPUxUHJ6HeMEBCoo6xTbr8QizXjBEHwWj3mZUha, address:104.36.82.21:4478): protocol: write tcp 159.223.143.39:36794->104.36.82.21:4478: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:49:31.910868793 (26h57m47.691802166s) [noise=true , address=104.36.82.22:4523    , piece_id=GUQYZN3KBIBMYA55CAZIYDRVT57QOMXQ2VQ55VLAQ6JQETDT7AIQ, seg_pos={0 0}, upload=25713]: Done uploading piece. err: ecclient: upload failed (node:1affHJR5pyytf5iLmsF2pd81qU3SK5F8JEUf47WqcshpD972LQ, address:104.36.82.22:4523): protocol: write tcp 159.223.143.39:34012->104.36.82.22:4523: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:49:31.910931136 (26h57m47.691864516s) [noise=true , address=104.36.82.40:4972    , piece_id=FLGCFR6Z5SIZRXCUTOKHRQ6S645VZZWSQYVFIWANCVXWEJ55AI4A, seg_pos={0 0}, upload=25713]: Done uploading piece. err: ecclient: upload failed (node:1XSuvCvq716GAa9WWmM3n2tqnfh8paaFUhjBzVzpiqwJtNifnr, address:104.36.82.40:4972): protocol: write tcp 159.223.143.39:53128->104.36.82.40:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:49:31.911002664 (26h57m47.691936032s) [noise=true , address=104.36.82.46:4505    , piece_id=NAEHGX44QPZ5N363FU52G6LADYXQDDXAASTDT3POYRGS6U774LSQ, seg_pos={0 0}, upload=25713]: Done uploading piece. err: ecclient: upload failed (node:1i6itGoBhLsLiqC8et8Y5L62fLjcWxy18u39NAr4R3nXNMWcsf, address:104.36.82.46:4505): protocol: write tcp 159.223.143.39:35458->104.36.82.46:4505: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:49:31.911093516 (26h57m47.692026896s) [noise=true , address=104.36.82.53:4130    , piece_id=KYJ3RO6BBJWUD7VOS2HSECXQHPD52QRDTVBSPYFLL7GWBUXBK5MQ, seg_pos={0 0}, upload=25713]: Done uploading piece. err: ecclient: upload failed (node:12N8MGboFyEV381rkrBva3G3advz3QQEY2HCHxwqzKX2X5zxNvb, address:104.36.82.53:4130): protocol: write tcp 159.223.143.39:32794->104.36.82.53:4130: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:49:31.911180723 (26h57m47.692114085s) [noise=true , address=104.36.82.3:4769     , piece_id=G2PHICQEEEF4H6PQRLH3LQBWIYZ4BKJHTKFN56S4CCDSXJTVBTPA, seg_pos={0 0}, upload=25713]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:104.36.82.3:4769): protocol: write tcp 159.223.143.39:51718->104.36.82.3:4769: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:49:31.912862686 (26h57m47.693796066s) [noise=true , address=104.36.82.10:4083    , piece_id=IOMZFD47YOMVLALTP3MC5EXXDB5K2Y3V37C5XIMSUIEFOANURW5A, seg_pos={0 0}, upload=25713]: Done uploading piece. err: ecclient: upload failed (node:12KZ5a6CQ9tUTrC6EYE7M8yg6p9fK95CN8zQLQmP4tdyz4rBfGh, address:104.36.82.10:4083): protocol: write tcp 159.223.143.39:53930->104.36.82.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:49:31.912952074 (26h57m47.693885436s) [noise=true , address=104.36.82.14:4246    , piece_id=Z476KF7P7SEUR4O237EDCLUPSWGWQLK3J4PP6A6R3T2NITUC3OXA, seg_pos={0 0}, upload=25713]: Done uploading piece. err: ecclient: upload failed (node:12dqLB6LVQJaSdEfjKmu5nXqRST733hn5rTxCcMQS8YJVv8Hwdc, address:104.36.82.14:4246): protocol: write tcp 159.223.143.39:59558->104.36.82.14:4246: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:49:31.913020154 (26h57m47.69395353s) [noise=true , address=104.36.82.50:4014    , piece_id=BTDWTVSX2SGELT6DW2XH4TGW72IQYGZ4UDO5WAVV73UQP2UY3WQQ, seg_pos={0 0}, upload=25713]: Done uploading piece. err: ecclient: upload failed (node:122mrzG2jkLjNdnrDVr6UMM2aw3wNNTaJMnNaUKdN8UwFUqLwVd, address:104.36.82.50:4014): protocol: write tcp 159.223.143.39:43350->104.36.82.50:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:49:31.913084085 (26h57m47.694017457s) [noise=true , address=104.36.82.56:4260    , piece_id=OEBSMGMSTBXJBN5KHJ2FTRWQNJJPG6MY7S6F55PVDLTGP7KSMLWA, seg_pos={0 0}, upload=25713]: Done uploading piece. err: ecclient: upload failed (node:1NjKYY18P7vTepAAujSj8Gep3STrHBdAdwaQ4VS6noje1oqSCH, address:104.36.82.56:4260): protocol: write tcp 159.223.143.39:46160->104.36.82.56:4260: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:49:31.913139275 (26h57m47.694072642s) [noise=true , address=104.36.82.52:4089    , piece_id=WLIQJQ7YURGB7JHGTLODXPZVJG24FB6RY2L5JM74UNXLDYORDJMA, seg_pos={0 0}, upload=25713]: Done uploading piece. err: ecclient: upload failed (node:12L2DTyJJ3v7KRjA424PVwQn9mpeq5Pq4ZTCerhrStgqgc3cVFk, address:104.36.82.52:4089): protocol: write tcp 159.223.143.39:35338->104.36.82.52:4089: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:49:31.913217825 (26h57m47.694151189s) [noise=true , address=104.36.82.13:4210    , piece_id=FAEFTTKDGQLBP3RBFWUCFYDTMLDGIKXKX56ISSEOWZEXAXX5ZETA, seg_pos={0 0}, upload=25713]: Done uploading piece. err: ecclient: upload failed (node:1ybrcx8SPgyEsJWq2P1KYaTwTj5v5BfENKNEZZxaKRvKvuqTiL, address:104.36.82.13:4210): protocol: context deadline exceeded; context deadline exceeded
Jul 28 00:49:31.913283158 (26h57m47.694216519s) [noise=true , address=104.36.82.47:4552    , piece_id=GED27X4HDTAYFKGSLSJPM54O2ZA5YJMVHADC26WTHISXRYVU5KFQ, seg_pos={0 0}, upload=25713]: Done uploading piece. err: ecclient: upload failed (node:1hoXgBFoGKKknrobbqfArxNhrsiUHKiiVZGWCnvxDSgBcae8gw, address:104.36.82.47:4552): protocol: write tcp 159.223.143.39:54396->104.36.82.47:4552: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:49:31.913352884 (26h57m47.694286246s) [noise=true , address=104.36.82.42:4801    , piece_id=7BV4ZRAADGGZUKBCRVSNJNW7GEV6BJPDKSJBTVKNVRBOPZ2TQ5SQ, seg_pos={0 0}, upload=25713]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:104.36.82.42:4801): protocol: write tcp 159.223.143.39:36396->104.36.82.42:4801: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:49:31.913374276 (26h57m47.694307665s) [seg_pos={0 0}, upload=25713]: Done waiting for segment. successful: 3 optimal: 14 errs: <nil>
Jul 28 00:49:31.935801899 (26h57m47.716735267s) [upload=25713]: Done uploading segments...
Jul 28 00:49:31.935823902 (26h57m47.716757279s) [upload=25713]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 3)
Duration: 1m0.049729857s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip: 
6 out of 60 is failed (threshold=3) [............X...XX.......X...........X...................X..]
duration: 
381.261352ms
FAIL (2025-07-28 00:48:52 UTC) 
orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 381.308957ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
7 out of 60 is failed (threshold=3) [..........X........X.............XX...........X...X...X.....]
duration: 
375.792792ms
FAIL (2025-07-28 00:48:35 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
0}, upload=25651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 159.223.143.39:39890->104.36.82.7:4909: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:44:18.975782256 (26h52m34.756715638s) [seg_pos={0 0}, upload=25651]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 28 00:44:18.975871874 (26h52m34.756805277s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 28 00:44:18.975878645 (26h52m34.756812034s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 28 00:44:19.019856339 (26h52m34.800789708s) [upload=25651]: Done uploading segments...
Jul 28 00:44:19.019879905 (26h52m34.800813283s) [upload=25651]: Upload finished. err: <nil>
Jul 28 00:45:02.257192553 (26h53m18.03812594s) [upload=25659]: Starting upload
Jul 28 00:45:02.257306836 (26h53m18.038240216s) [upload=25659]: Done starting upload
Jul 28 00:45:02.257434360 (26h53m18.038367739s) [upload=25659]: Uploading segments...
Jul 28 00:45:02.257858888 (26h53m18.038792251s) [upload=25659]: Got next segment. Inline: false
Jul 28 00:45:02.257951856 (26h53m18.038885235s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 28 00:45:02.257959915 (26h53m18.038893293s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 28 00:45:02.293048419 (26h53m18.073981806s) [seg_pos={0 0}, upload=25659]: Begin upload segment...
Jul 28 00:45:02.293125122 (26h53m18.074058491s) [seg_pos={0 0}, upload=25659]: Done begin upload segment.
Jul 28 00:45:02.293171689 (26h53m18.074105073s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293724212 (26h53m18.074657618s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293744577 (26h53m18.074677964s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294292781 (26h53m18.075226212s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294343706 (26h53m18.075277094s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294756878 (26h53m18.075690269s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294891918 (26h53m18.075825306s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295221121 (26h53m18.076154513s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295558583 (26h53m18.076491975s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295604310 (26h53m18.0765377s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295991730 (26h53m18.07692512s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.296088221 (26h53m18.077021615s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.301695563 (26h53m18.082628952s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302203775 (26h53m18.083137164s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302629001 (26h53m18.083562389s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302995157 (26h53m18.083928524s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.418208178 (26h53m18.19914155s) [upload=25659]: Next returned nil segment
Jul 28 00:45:02.418226832 (26h53m18.19916021s) [upload=25659]: Waiting for error group managing segments...
Jul 28 00:45:02.424117363 (26h53m18.205050731s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.427234519 (26h53m18.208167896s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.430114806 (26h53m18.211048185s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.435624120 (26h53m18.216557505s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436119655 (26h53m18.217053027s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436405218 (26h53m18.217338582s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436887494 (26h53m18.217820863s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436998726 (26h53m18.217932114s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437527079 (26h53m18.218460459s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437625274 (26h53m18.218558642s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439469614 (26h53m18.220402995s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439557143 (26h53m18.220490506s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:32.221331076 (26h53m48.002264449s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: write tcp 159.223.143.39:33378->104.36.82.16:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221411781 (26h53m48.002345162s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221457184 (26h53m48.002390547s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: rpc: dial tcp 104.36.82.13:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221473904 (26h53m48.002407294s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:49546->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221516309 (26h53m48.002449671s) [seg_pos={0 0}, upload=25659]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul 28 00:45:32.251981717 (26h53m48.032915086s) [upload=25659]: Done uploading segments...
Jul 28 00:45:32.252005327 (26h53m48.032938704s) [upload=25659]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 375.814884ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)
antiblip: 
42 out of 60 is failed (threshold=3) [.................XXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration: 
1m0.028744382s
FAIL (2025-07-28 00:48:31 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 2)
======LOGS======
pos={0 0}, upload=25698]: Uploading piece...
Jul 28 00:47:31.839188525 (26h55m47.620121906s) [noise=true , address=104.36.82.44:4872    , piece_id=XKZ6PV47BWWNF4DG67ZOOYPI2R3FIDBGRRXUTSALFBNAHSDJF2NA, seg_pos={0 0}, upload=25698]: Uploading piece...
Jul 28 00:47:31.839502331 (26h55m47.62043571s) [noise=true , address=104.36.82.21:4503    , piece_id=HPQ7AE2PVLPSKM24DBGKDHFFDOYHRFLOLK6QJXNHTEZQ7JJTCFZA, seg_pos={0 0}, upload=25698]: Uploading piece...
Jul 28 00:47:31.839855179 (26h55m47.620788562s) [noise=true , address=104.36.82.46:4481    , piece_id=I3LFNXT7MLNL2F3MECMK23JOZHRA3PLICTYZRTP3WWMOEEOH7TAQ, seg_pos={0 0}, upload=25698]: Uploading piece...
Jul 28 00:47:31.840258207 (26h55m47.621191585s) [noise=true , address=104.36.82.5:4866     , piece_id=GER4JKPHVNS2PKEMW6LSKGKY4YEGNLHAQUGELH7ETMJI5YNZKK6A, seg_pos={0 0}, upload=25698]: Uploading piece...
Jul 28 00:47:31.840583635 (26h55m47.621517018s) [noise=true , address=79.127.201.215:4349  , piece_id=E4DGHTX66NRGBOOWUEBM4PIPHEBUEE7G3CMVAQOJXS67PRBFARIA, seg_pos={0 0}, upload=25698]: Uploading piece...
Jul 28 00:47:31.840934126 (26h55m47.621867509s) [noise=true , address=104.36.82.10:4091    , piece_id=XRTL4K6UBXRNYTMW7GRZF6AFXQ5QBSS577MB3BHHDY2OHFMKJODA, seg_pos={0 0}, upload=25698]: Uploading piece...
Jul 28 00:47:31.841345667 (26h55m47.622279044s) [noise=true , address=104.36.82.52:4097    , piece_id=FTHOO6Y4PRZ2SHZCQSOORYXNV6FGT3HNT4FYWLJRVUPN6DEESX4Q, seg_pos={0 0}, upload=25698]: Uploading piece...
Jul 28 00:47:31.922800841 (26h55m47.70373421s) [upload=25698]: Next returned nil segment
Jul 28 00:47:31.922818204 (26h55m47.703751634s) [upload=25698]: Waiting for error group managing segments...
Jul 28 00:47:31.970665091 (26h55m47.751598459s) [noise=true , address=104.36.82.5:4866     , piece_id=GER4JKPHVNS2PKEMW6LSKGKY4YEGNLHAQUGELH7ETMJI5YNZKK6A, seg_pos={0 0}, upload=25698]: Done uploading piece. err: <nil>
Jul 28 00:47:35.075514503 (26h55m50.85644789s) [noise=true , address=79.127.201.215:4349  , piece_id=E4DGHTX66NRGBOOWUEBM4PIPHEBUEE7G3CMVAQOJXS67PRBFARIA, seg_pos={0 0}, upload=25698]: Done uploading piece. err: <nil>
Jul 28 00:48:31.770633225 (26h56m47.551566592s) [noise=true , address=104.36.82.50:4028    , piece_id=RDQUU2CNTRYGBXEVNTWSRVMAVFUKXQ4V26STJV4M6YDZE7PWCU3Q, seg_pos={0 0}, upload=25698]: Done uploading piece. err: ecclient: upload failed (node:12B8N15ijqmYJVmjCT74JTLcMEYSZ4xdAWxVqRRF6w8hgYJkmHF, address:104.36.82.50:4028): protocol: rpc: dial tcp 104.36.82.50:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:48:31.770691622 (26h56m47.551624984s) [noise=true , address=104.36.82.44:4872    , piece_id=XKZ6PV47BWWNF4DG67ZOOYPI2R3FIDBGRRXUTSALFBNAHSDJF2NA, seg_pos={0 0}, upload=25698]: Done uploading piece. err: ecclient: upload failed (node:12dNdm2kAok8jFgbYE24oXUL9KCJVDDiinLUJgDSYEY3fTEBJec, address:104.36.82.44:4872): protocol: write tcp 159.223.143.39:34194->104.36.82.44:4872: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:48:31.770735348 (26h56m47.551668724s) [noise=true , address=104.36.82.56:4238    , piece_id=WO6ZMIXLFX53WDZYUL2NR25Z7W45HOH7I3SD2IRX5Q2U4ZABJWBA, seg_pos={0 0}, upload=25698]: Done uploading piece. err: ecclient: upload failed (node:12U2euZtUdrPZGo2PieqDnKtSftgtfdTRk6V8VfYS4yUW5z3qM5, address:104.36.82.56:4238): protocol: rpc: dial tcp 104.36.82.56:4238: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:48:31.770789520 (26h56m47.551722881s) [noise=true , address=104.36.82.20:4439    , piece_id=OMD67HWZDXMFJ6PJHKKU3OJSWP5T2XJ6YMJVNUG3W4G3MDPEZ2KQ, seg_pos={0 0}, upload=25698]: Done uploading piece. err: ecclient: upload failed (node:1wgkbc3Mkej7supxhYMtUdBvn9t45i6kmAokAUkqFxEpehMjAn, address:104.36.82.20:4439): protocol: write tcp 159.223.143.39:35388->104.36.82.20:4439: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:48:31.770873829 (26h56m47.551807191s) [noise=true , address=104.36.82.13:4205    , piece_id=WBPM357YJZQJJYBL5UFPQUFJE7NGNRHH3BKGBVZEKZKZNAOTOOLQ, seg_pos={0 0}, upload=25698]: Done uploading piece. err: ecclient: upload failed (node:12pih2cgjPo5YeP6B7g8diM7p11VRJfHy5H34GUT9CK77W2cTLr, address:104.36.82.13:4205): protocol: write tcp 159.223.143.39:39138->104.36.82.13:4205: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:48:31.770888185 (26h56m47.551821561s) [noise=true , address=104.36.82.52:4097    , piece_id=FTHOO6Y4PRZ2SHZCQSOORYXNV6FGT3HNT4FYWLJRVUPN6DEESX4Q, seg_pos={0 0}, upload=25698]: Done uploading piece. err: ecclient: upload failed (node:1sMhTktqf2H4aZSyFV3m9Z6eMVxL2U52bvUHVHUh2gUCVp32AM, address:104.36.82.52:4097): protocol: rpc: dial tcp 104.36.82.52:4097: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:48:31.770939938 (26h56m47.551873313s) [noise=true , address=104.36.82.54:4177    , piece_id=WJ3YBOCEFMA56XUXXRTRJQFIC72RAAGIVRYQWUPJ2WBRTWYAGHNA, seg_pos={0 0}, upload=25698]: Done uploading piece. err: ecclient: upload failed (node:1bhN3cR22QuAYihnnB3x3R97UZRWJ8sFPP8fjQcXZ7qhNZphZG, address:104.36.82.54:4177): protocol: rpc: dial tcp 104.36.82.54:4177: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:48:31.770964621 (26h56m47.551897992s) [noise=true , address=104.36.82.46:4481    , piece_id=I3LFNXT7MLNL2F3MECMK23JOZHRA3PLICTYZRTP3WWMOEEOH7TAQ, seg_pos={0 0}, upload=25698]: Done uploading piece. err: ecclient: upload failed (node:12bRK8WWmAnWUwzCfHu8JtZgrruYSJCvhnsYTKcaLccJcv5RQjp, address:104.36.82.46:4481): protocol: write tcp 159.223.143.39:36938->104.36.82.46:4481: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:48:31.771006178 (26h56m47.551939554s) [noise=true , address=104.36.82.21:4487    , piece_id=VOUUMCBPNU6KUSC2GT5PCVMMYFRKSKZQZYDV5K3OWWHSZGSNFKYQ, seg_pos={0 0}, upload=25698]: Done uploading piece. err: ecclient: upload failed (node:19xsJiapoS3bYRaeQoZ3w6U9eKWTkGpncNPifT9MAqnxCeL7yK, address:104.36.82.21:4487): protocol: rpc: dial tcp 104.36.82.21:4487: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:48:31.771073413 (26h56m47.552006779s) [noise=true , address=104.36.82.10:4091    , piece_id=XRTL4K6UBXRNYTMW7GRZF6AFXQ5QBSS577MB3BHHDY2OHFMKJODA, seg_pos={0 0}, upload=25698]: Done uploading piece. err: ecclient: upload failed (node:1yyUzkigTS9m3bDrZtAFL7tciNxpiCNswBSfBqXn1UW83GMLwi, address:104.36.82.10:4091): protocol: write tcp 159.223.143.39:42692->104.36.82.10:4091: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:48:31.771087843 (26h56m47.552021217s) [noise=true , address=104.36.82.47:4527    , piece_id=ZLTZB5ASG3O7V4NZUJN34CLZFAZVBZOZ7RRKQ7AUQEKKQCKOFEOA, seg_pos={0 0}, upload=25698]: Done uploading piece. err: ecclient: upload failed (node:17xmkbFoj3PTJnAygsjRAcfdd17AfSkPP2xdZp68GGPrvfMuU6, address:104.36.82.47:4527): protocol: write tcp 159.223.143.39:54992->104.36.82.47:4527: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:48:31.771143275 (26h56m47.55207664s) [noise=true , address=104.36.82.53:4144    , piece_id=ECDFT54ELEFFYUM3RMCOUU5SHCMUVVI3WTZARBBVYEYSRL2I4OHA, seg_pos={0 0}, upload=25698]: Done uploading piece. err: ecclient: upload failed (node:1FfzcPNLmNdrvoZBKcmCLz8d9rYSAnjFTo4NBnxBzN6dVcTNwt, address:104.36.82.53:4144): protocol: write tcp 159.223.143.39:35098->104.36.82.53:4144: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:48:31.771153271 (26h56m47.552086648s) [noise=true , address=104.36.82.4:4818     , piece_id=E7Y2WN3MEHZWLP34XFC72GDK6X3XMNIK3L36S64QTSKYGRUNA7MA, seg_pos={0 0}, upload=25698]: Done uploading piece. err: ecclient: upload failed (node:1235eFkz7CFwwiM8UUZU1vnqDneEic7RpFzeTU46gvTMZWhhox8, address:104.36.82.4:4818): protocol: write tcp 159.223.143.39:47564->104.36.82.4:4818: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:48:31.771216060 (26h56m47.552149439s) [noise=true , address=104.36.82.3:4788     , piece_id=GESRTDGZT7WXCWGORX7QO2MKKALJPMXSK4BEZSTMK3XO6KWLRQYQ, seg_pos={0 0}, upload=25698]: Done uploading piece. err: ecclient: upload failed (node:127EfVqSahKLLy8nF95kPd1a39xfThK4AKJ2X2XteKVCNntcMD4, address:104.36.82.3:4788): protocol: write tcp 159.223.143.39:37530->104.36.82.3:4788: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:48:31.771301349 (26h56m47.552234714s) [noise=true , address=104.36.82.11:4119    , piece_id=A6HR7ATFMMZRC3RA7V5LJCDR3XDMPKOMILSZT7MNLNR3KVTBZFCA, seg_pos={0 0}, upload=25698]: Done uploading piece. err: ecclient: upload failed (node:121xNAnkm9HZXUszp26UmWTsuhp3igyeDVea6gSCdtrQUQDVHpW, address:104.36.82.11:4119): protocol: write tcp 159.223.143.39:48150->104.36.82.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:48:31.771337808 (26h56m47.552271183s) [noise=true , address=104.36.82.21:4503    , piece_id=HPQ7AE2PVLPSKM24DBGKDHFFDOYHRFLOLK6QJXNHTEZQ7JJTCFZA, seg_pos={0 0}, upload=25698]: Done uploading piece. err: ecclient: upload failed (node:17F3Gn5LYgekp8WgagEdFXtRu8jCc27qA9UuW7pWsJ9FUFKFSi, address:104.36.82.21:4503): protocol: write tcp 159.223.143.39:49426->104.36.82.21:4503: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:48:31.771383498 (26h56m47.552316885s) [seg_pos={0 0}, upload=25698]: Done waiting for segment. successful: 2 optimal: 14 errs: <nil>
Jul 28 00:48:31.796127519 (26h56m47.577060897s) [upload=25698]: Done uploading segments...
Jul 28 00:48:31.796156920 (26h56m47.577090295s) [upload=25698]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 2)
Duration: 1m0.028760431s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip: 
6 out of 60 is failed (threshold=3) [.............X...XX.......X...........X...................X.]
duration: 
388.175579ms
FAIL (2025-07-28 00:47:59 UTC) 
orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 388.224682ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip: 
6 out of 60 is failed (threshold=3) [..............X...XX.......X...........X...................X]
duration: 
30.034700903s
FAIL (2025-07-28 00:47:53 UTC) 
orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 30.034735978s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
7 out of 60 is failed (threshold=3) [...........X........X.............XX...........X...X...X....]
duration: 
495.260583ms
FAIL (2025-07-28 00:47:44 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
0}, upload=25651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 159.223.143.39:39890->104.36.82.7:4909: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:44:18.975782256 (26h52m34.756715638s) [seg_pos={0 0}, upload=25651]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 28 00:44:18.975871874 (26h52m34.756805277s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 28 00:44:18.975878645 (26h52m34.756812034s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 28 00:44:19.019856339 (26h52m34.800789708s) [upload=25651]: Done uploading segments...
Jul 28 00:44:19.019879905 (26h52m34.800813283s) [upload=25651]: Upload finished. err: <nil>
Jul 28 00:45:02.257192553 (26h53m18.03812594s) [upload=25659]: Starting upload
Jul 28 00:45:02.257306836 (26h53m18.038240216s) [upload=25659]: Done starting upload
Jul 28 00:45:02.257434360 (26h53m18.038367739s) [upload=25659]: Uploading segments...
Jul 28 00:45:02.257858888 (26h53m18.038792251s) [upload=25659]: Got next segment. Inline: false
Jul 28 00:45:02.257951856 (26h53m18.038885235s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 28 00:45:02.257959915 (26h53m18.038893293s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 28 00:45:02.293048419 (26h53m18.073981806s) [seg_pos={0 0}, upload=25659]: Begin upload segment...
Jul 28 00:45:02.293125122 (26h53m18.074058491s) [seg_pos={0 0}, upload=25659]: Done begin upload segment.
Jul 28 00:45:02.293171689 (26h53m18.074105073s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293724212 (26h53m18.074657618s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293744577 (26h53m18.074677964s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294292781 (26h53m18.075226212s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294343706 (26h53m18.075277094s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294756878 (26h53m18.075690269s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294891918 (26h53m18.075825306s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295221121 (26h53m18.076154513s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295558583 (26h53m18.076491975s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295604310 (26h53m18.0765377s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295991730 (26h53m18.07692512s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.296088221 (26h53m18.077021615s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.301695563 (26h53m18.082628952s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302203775 (26h53m18.083137164s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302629001 (26h53m18.083562389s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302995157 (26h53m18.083928524s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.418208178 (26h53m18.19914155s) [upload=25659]: Next returned nil segment
Jul 28 00:45:02.418226832 (26h53m18.19916021s) [upload=25659]: Waiting for error group managing segments...
Jul 28 00:45:02.424117363 (26h53m18.205050731s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.427234519 (26h53m18.208167896s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.430114806 (26h53m18.211048185s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.435624120 (26h53m18.216557505s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436119655 (26h53m18.217053027s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436405218 (26h53m18.217338582s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436887494 (26h53m18.217820863s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436998726 (26h53m18.217932114s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437527079 (26h53m18.218460459s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437625274 (26h53m18.218558642s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439469614 (26h53m18.220402995s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439557143 (26h53m18.220490506s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:32.221331076 (26h53m48.002264449s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: write tcp 159.223.143.39:33378->104.36.82.16:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221411781 (26h53m48.002345162s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221457184 (26h53m48.002390547s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: rpc: dial tcp 104.36.82.13:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221473904 (26h53m48.002407294s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:49546->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221516309 (26h53m48.002449671s) [seg_pos={0 0}, upload=25659]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul 28 00:45:32.251981717 (26h53m48.032915086s) [upload=25659]: Done uploading segments...
Jul 28 00:45:32.252005327 (26h53m48.032938704s) [upload=25659]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 495.303237ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)
antiblip: 
41 out of 60 is failed (threshold=3) [..................XXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration: 
1m0.025575168s
FAIL (2025-07-28 00:47:31 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 3)
======LOGS======
4DVKQHJUH3DNSQ, seg_pos={0 0}, upload=25680]: Uploading piece...
Jul 28 00:46:31.726088950 (26h54m47.507022327s) [noise=true , address=104.36.82.13:4214    , piece_id=RV2DQFGZQHPODOYQPA7QSLL5UUOI2S4VBMINX6HWLCROKAI3KNBQ, seg_pos={0 0}, upload=25680]: Uploading piece...
Jul 28 00:46:31.726718225 (26h54m47.507651601s) [noise=true , address=104.36.82.14:4225    , piece_id=4FQJMU2WXHMLDBC7DMJCCIV5XHCSV6IL65ANBANJEWUB64ZBHVGA, seg_pos={0 0}, upload=25680]: Uploading piece...
Jul 28 00:46:31.727605130 (26h54m47.508538498s) [noise=true , address=104.36.82.53:4127    , piece_id=ASZCUEN45K6X7XPGJ3R3HV37KRJ36H2A32X4JV5DUR4TB6Q4VQIA, seg_pos={0 0}, upload=25680]: Uploading piece...
Jul 28 00:46:31.728220317 (26h54m47.509153692s) [noise=true , address=79.127.201.215:4346  , piece_id=SL2K6CYYBCRIO42ZQ52ABKB7DV325PJRGMLO4GA5V656MU5KAUFQ, seg_pos={0 0}, upload=25680]: Uploading piece...
Jul 28 00:46:31.731768674 (26h54m47.512702035s) [noise=true , address=104.36.82.21:4475    , piece_id=HWSIHZZSVZJ6J3KGYNMSMFJRLCRCKDNQESEXMAJRKLXEINZWRXKQ, seg_pos={0 0}, upload=25680]: Uploading piece...
Jul 28 00:46:31.732237255 (26h54m47.513170623s) [noise=true , address=104.36.82.3:4794     , piece_id=ZKQJOXRC67CN6TO3G3CO6B5ALU77TOPZ3S6UB7G2WU62MWRP3JGA, seg_pos={0 0}, upload=25680]: Uploading piece...
Jul 28 00:46:31.732446529 (26h54m47.513379898s) [noise=true , address=104.36.82.54:4168    , piece_id=L7NJLQLBUED4OWEKNUSURGKSV4SVVEQQF5CTVVJM3UPC537Q77OQ, seg_pos={0 0}, upload=25680]: Uploading piece...
Jul 28 00:46:31.733654484 (26h54m47.514587854s) [noise=true , address=104.36.82.9:4020     , piece_id=3M5O755EIJB4VGQWSCM665MVP7FQIDK5EUA5XNRFPMFTXLUUAONQ, seg_pos={0 0}, upload=25680]: Uploading piece...
Jul 28 00:46:31.930734404 (26h54m47.711667786s) [upload=25680]: Next returned nil segment
Jul 28 00:46:31.930756927 (26h54m47.711690312s) [upload=25680]: Waiting for error group managing segments...
Jul 28 00:46:32.495543644 (26h54m48.276477032s) [noise=true , address=64.49.13.137:4498    , piece_id=V7UL5UZXSZV3LFXYFRZBSXXCV6FXPRX47XTMMEPRJSAC47RFH7AA, seg_pos={0 0}, upload=25680]: Done uploading piece. err: <nil>
Jul 28 00:46:42.140397873 (26h54m57.921331236s) [noise=true , address=79.127.201.215:4346  , piece_id=SL2K6CYYBCRIO42ZQ52ABKB7DV325PJRGMLO4GA5V656MU5KAUFQ, seg_pos={0 0}, upload=25680]: Done uploading piece. err: <nil>
Jul 28 00:46:43.223434900 (26h54m59.00436828s) [noise=true , address=79.127.201.215:4344  , piece_id=XAUUUL74ADNMV4VR6R3ILAH5W3N7ASTVMI4EPFRMFDMY2BVM2LAA, seg_pos={0 0}, upload=25680]: Done uploading piece. err: <nil>
Jul 28 00:47:31.656445936 (26h55m47.4373793s) [noise=true , address=104.36.82.55:4202    , piece_id=ZEC3YAAEFVFKK6N4V5OSL737MNNJB7WCXRC2A24DJ7VQ5TA5XHJA, seg_pos={0 0}, upload=25680]: Done uploading piece. err: ecclient: upload failed (node:121ogdaiVGybXEVsS8LDJuJRYXGxb1LCNnGPuwbde96eSEzkTW4, address:104.36.82.55:4202): protocol: rpc: dial tcp 104.36.82.55:4202: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:47:31.656559817 (26h55m47.437493192s) [noise=true , address=104.36.82.54:4168    , piece_id=L7NJLQLBUED4OWEKNUSURGKSV4SVVEQQF5CTVVJM3UPC537Q77OQ, seg_pos={0 0}, upload=25680]: Done uploading piece. err: ecclient: upload failed (node:1XH68Yk8yh2Qa8vcPRYAuLHdGR7Jy4KtSBKYKgwAJpdDvVu7up, address:104.36.82.54:4168): protocol: write tcp 159.223.143.39:33964->104.36.82.54:4168: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:47:31.656624848 (26h55m47.437558224s) [noise=true , address=104.36.82.44:4885    , piece_id=DC624FKJ5PZPGFMBJLAK7BCKMVD7AKKOBHWGIOFMSHF5EZE477CA, seg_pos={0 0}, upload=25680]: Done uploading piece. err: ecclient: upload failed (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG, address:104.36.82.44:4885): protocol: write tcp 159.223.143.39:54760->104.36.82.44:4885: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:47:31.656643523 (26h55m47.43757689s) [noise=true , address=104.36.82.3:4794     , piece_id=ZKQJOXRC67CN6TO3G3CO6B5ALU77TOPZ3S6UB7G2WU62MWRP3JGA, seg_pos={0 0}, upload=25680]: Done uploading piece. err: ecclient: upload failed (node:12ctLLD5skk4t5i3BFYgoTmuZ2aqsX23wDGDMnKriQ7U2KXyPAo, address:104.36.82.3:4794): protocol: rpc: dial tcp 104.36.82.3:4794: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:47:31.656680536 (26h55m47.437613912s) [noise=true , address=104.36.82.14:4225    , piece_id=4FQJMU2WXHMLDBC7DMJCCIV5XHCSV6IL65ANBANJEWUB64ZBHVGA, seg_pos={0 0}, upload=25680]: Done uploading piece. err: ecclient: upload failed (node:12QtY3yN6J1H8htRneePHu4RmbN7A1xkgywphMDVsVxhTczGupY, address:104.36.82.14:4225): protocol: write tcp 159.223.143.39:56950->104.36.82.14:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:47:31.656728121 (26h55m47.437661488s) [noise=true , address=104.36.82.9:4020     , piece_id=3M5O755EIJB4VGQWSCM665MVP7FQIDK5EUA5XNRFPMFTXLUUAONQ, seg_pos={0 0}, upload=25680]: Done uploading piece. err: ecclient: upload failed (node:12XXPyMaA6o5DFhMTkkWf14bdqZoZ86MNjxzfSqvNU2KQj8xSyv, address:104.36.82.9:4020): protocol: rpc: dial tcp 104.36.82.9:4020: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:47:31.656781663 (26h55m47.437715039s) [noise=true , address=104.36.82.47:4547    , piece_id=Q4HANHPEM5Y5WPTQCBNRLCSV24MNWRBUYKUF2WLQIITRSAVVHGYA, seg_pos={0 0}, upload=25680]: Done uploading piece. err: ecclient: upload failed (node:12brQg9WQMKJgp1buz1wMS8W11XEwAx7RHG5h51EmLvwY5D5Hky, address:104.36.82.47:4547): protocol: write tcp 159.223.143.39:44608->104.36.82.47:4547: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:47:31.656814805 (26h55m47.437748174s) [noise=true , address=104.36.82.15:4289    , piece_id=WLOYUNAMAL6VDUJ6XHFWHHXQ54S363L42QOEUU2576RXTV6OKT2A, seg_pos={0 0}, upload=25680]: Done uploading piece. err: ecclient: upload failed (node:126WPF6MtX7YjuKf73rGJrifdXyd78KY4FdL757qDipGNHBW6uY, address:104.36.82.15:4289): protocol: write tcp 159.223.143.39:48388->104.36.82.15:4289: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:47:31.656827858 (26h55m47.437761226s) [noise=true , address=104.36.82.56:4260    , piece_id=GMMWXYBOOHIO2WGY3FXLXD6KXWEMMBWINNKTQZ747HHLSOPPBYIA, seg_pos={0 0}, upload=25680]: Done uploading piece. err: ecclient: upload failed (node:1NjKYY18P7vTepAAujSj8Gep3STrHBdAdwaQ4VS6noje1oqSCH, address:104.36.82.56:4260): protocol: write tcp 159.223.143.39:38214->104.36.82.56:4260: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:47:31.656840262 (26h55m47.437773638s) [noise=true , address=104.36.82.46:4491    , piece_id=NGTOC2CP6I6E6WI5YAV4KQSPC7W4CQ57MIQDXR4DVKQHJUH3DNSQ, seg_pos={0 0}, upload=25680]: Done uploading piece. err: ecclient: upload failed (node:1CspvuKb3CWXrAW8FtFEoCVUNhLzWsbpwpW5uUfgyWy71R3qYX, address:104.36.82.46:4491): protocol: write tcp 159.223.143.39:57968->104.36.82.46:4491: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:47:31.656889601 (26h55m47.437822976s) [noise=true , address=104.36.82.13:4214    , piece_id=RV2DQFGZQHPODOYQPA7QSLL5UUOI2S4VBMINX6HWLCROKAI3KNBQ, seg_pos={0 0}, upload=25680]: Done uploading piece. err: ecclient: upload failed (node:1nJ8uqwxf6x4hX52AjAdCbMKi1BKGGbMvrQbiazJdLpn6j69ay, address:104.36.82.13:4214): protocol: write tcp 159.223.143.39:40446->104.36.82.13:4214: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:47:31.656926215 (26h55m47.437859593s) [noise=true , address=104.36.82.15:4283    , piece_id=6NP3OZX5T6H2CG6QX6PCY3Z7R3FQB6MICVNYZITBQUCEBPOIINRQ, seg_pos={0 0}, upload=25680]: Done uploading piece. err: ecclient: upload failed (node:1ytKjZnmERiZtgJuMJhXG5KFwUQwie27odve4LPE7Pw5hR5ZaU, address:104.36.82.15:4283): protocol: write tcp 159.223.143.39:36314->104.36.82.15:4283: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:47:31.656940581 (26h55m47.437873956s) [noise=true , address=104.36.82.21:4475    , piece_id=HWSIHZZSVZJ6J3KGYNMSMFJRLCRCKDNQESEXMAJRKLXEINZWRXKQ, seg_pos={0 0}, upload=25680]: Done uploading piece. err: ecclient: upload failed (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk, address:104.36.82.21:4475): protocol: write tcp 159.223.143.39:54812->104.36.82.21:4475: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:47:31.656949997 (26h55m47.437883365s) [noise=true , address=104.36.82.53:4127    , piece_id=ASZCUEN45K6X7XPGJ3R3HV37KRJ36H2A32X4JV5DUR4TB6Q4VQIA, seg_pos={0 0}, upload=25680]: Done uploading piece. err: ecclient: upload failed (node:12VM12t6Dq6SyApviDYSSUqWnnzW3wTfA4o8VsvJaah4fLj9EWh, address:104.36.82.53:4127): protocol: rpc: dial tcp 104.36.82.53:4127: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:47:31.656994243 (26h55m47.43792762s) [noise=true , address=104.36.82.52:4106    , piece_id=6WNHQ4SIYF2VEK3Q5O76Y6JDHYLCLW2F663WS42DZRKRIF2V3AWA, seg_pos={0 0}, upload=25680]: Done uploading piece. err: ecclient: upload failed (node:12E2hMuAUbJyh5F4yey1EtRLYhzr29wxs47qY3zGxmGvFNUctk2, address:104.36.82.52:4106): protocol: write tcp 159.223.143.39:42860->104.36.82.52:4106: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:47:31.657033472 (26h55m47.437966857s) [seg_pos={0 0}, upload=25680]: Done waiting for segment. successful: 3 optimal: 14 errs: <nil>
Jul 28 00:47:31.678969657 (26h55m47.45990302s) [upload=25680]: Done uploading segments...
Jul 28 00:47:31.678999804 (26h55m47.45993317s) [upload=25680]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 3)
Duration: 1m0.025594024s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
7 out of 60 is failed (threshold=3) [............X........X.............XX...........X...X...X...]
duration: 
366.697068ms
FAIL (2025-07-28 00:47:15 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
0}, upload=25651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 159.223.143.39:39890->104.36.82.7:4909: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:44:18.975782256 (26h52m34.756715638s) [seg_pos={0 0}, upload=25651]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 28 00:44:18.975871874 (26h52m34.756805277s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 28 00:44:18.975878645 (26h52m34.756812034s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 28 00:44:19.019856339 (26h52m34.800789708s) [upload=25651]: Done uploading segments...
Jul 28 00:44:19.019879905 (26h52m34.800813283s) [upload=25651]: Upload finished. err: <nil>
Jul 28 00:45:02.257192553 (26h53m18.03812594s) [upload=25659]: Starting upload
Jul 28 00:45:02.257306836 (26h53m18.038240216s) [upload=25659]: Done starting upload
Jul 28 00:45:02.257434360 (26h53m18.038367739s) [upload=25659]: Uploading segments...
Jul 28 00:45:02.257858888 (26h53m18.038792251s) [upload=25659]: Got next segment. Inline: false
Jul 28 00:45:02.257951856 (26h53m18.038885235s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 28 00:45:02.257959915 (26h53m18.038893293s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 28 00:45:02.293048419 (26h53m18.073981806s) [seg_pos={0 0}, upload=25659]: Begin upload segment...
Jul 28 00:45:02.293125122 (26h53m18.074058491s) [seg_pos={0 0}, upload=25659]: Done begin upload segment.
Jul 28 00:45:02.293171689 (26h53m18.074105073s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293724212 (26h53m18.074657618s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293744577 (26h53m18.074677964s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294292781 (26h53m18.075226212s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294343706 (26h53m18.075277094s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294756878 (26h53m18.075690269s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294891918 (26h53m18.075825306s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295221121 (26h53m18.076154513s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295558583 (26h53m18.076491975s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295604310 (26h53m18.0765377s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295991730 (26h53m18.07692512s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.296088221 (26h53m18.077021615s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.301695563 (26h53m18.082628952s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302203775 (26h53m18.083137164s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302629001 (26h53m18.083562389s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302995157 (26h53m18.083928524s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.418208178 (26h53m18.19914155s) [upload=25659]: Next returned nil segment
Jul 28 00:45:02.418226832 (26h53m18.19916021s) [upload=25659]: Waiting for error group managing segments...
Jul 28 00:45:02.424117363 (26h53m18.205050731s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.427234519 (26h53m18.208167896s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.430114806 (26h53m18.211048185s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.435624120 (26h53m18.216557505s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436119655 (26h53m18.217053027s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436405218 (26h53m18.217338582s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436887494 (26h53m18.217820863s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436998726 (26h53m18.217932114s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437527079 (26h53m18.218460459s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437625274 (26h53m18.218558642s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439469614 (26h53m18.220402995s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439557143 (26h53m18.220490506s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:32.221331076 (26h53m48.002264449s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: write tcp 159.223.143.39:33378->104.36.82.16:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221411781 (26h53m48.002345162s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221457184 (26h53m48.002390547s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: rpc: dial tcp 104.36.82.13:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221473904 (26h53m48.002407294s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:49546->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221516309 (26h53m48.002449671s) [seg_pos={0 0}, upload=25659]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul 28 00:45:32.251981717 (26h53m48.032915086s) [upload=25659]: Done uploading segments...
Jul 28 00:45:32.252005327 (26h53m48.032938704s) [upload=25659]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 366.728763ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
7 out of 60 is failed (threshold=3) [.............X........X.............XX...........X...X...X..]
duration: 
581.747493ms
FAIL (2025-07-28 00:46:44 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
0}, upload=25651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 159.223.143.39:39890->104.36.82.7:4909: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:44:18.975782256 (26h52m34.756715638s) [seg_pos={0 0}, upload=25651]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 28 00:44:18.975871874 (26h52m34.756805277s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 28 00:44:18.975878645 (26h52m34.756812034s) [upload=25651]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 28 00:44:19.019856339 (26h52m34.800789708s) [upload=25651]: Done uploading segments...
Jul 28 00:44:19.019879905 (26h52m34.800813283s) [upload=25651]: Upload finished. err: <nil>
Jul 28 00:45:02.257192553 (26h53m18.03812594s) [upload=25659]: Starting upload
Jul 28 00:45:02.257306836 (26h53m18.038240216s) [upload=25659]: Done starting upload
Jul 28 00:45:02.257434360 (26h53m18.038367739s) [upload=25659]: Uploading segments...
Jul 28 00:45:02.257858888 (26h53m18.038792251s) [upload=25659]: Got next segment. Inline: false
Jul 28 00:45:02.257951856 (26h53m18.038885235s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 28 00:45:02.257959915 (26h53m18.038893293s) [upload=25659]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 28 00:45:02.293048419 (26h53m18.073981806s) [seg_pos={0 0}, upload=25659]: Begin upload segment...
Jul 28 00:45:02.293125122 (26h53m18.074058491s) [seg_pos={0 0}, upload=25659]: Done begin upload segment.
Jul 28 00:45:02.293171689 (26h53m18.074105073s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293724212 (26h53m18.074657618s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.293744577 (26h53m18.074677964s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294292781 (26h53m18.075226212s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294343706 (26h53m18.075277094s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294756878 (26h53m18.075690269s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.294891918 (26h53m18.075825306s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295221121 (26h53m18.076154513s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295558583 (26h53m18.076491975s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295604310 (26h53m18.0765377s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.295991730 (26h53m18.07692512s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.296088221 (26h53m18.077021615s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.301695563 (26h53m18.082628952s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302203775 (26h53m18.083137164s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302629001 (26h53m18.083562389s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.302995157 (26h53m18.083928524s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Uploading piece...
Jul 28 00:45:02.418208178 (26h53m18.19914155s) [upload=25659]: Next returned nil segment
Jul 28 00:45:02.418226832 (26h53m18.19916021s) [upload=25659]: Waiting for error group managing segments...
Jul 28 00:45:02.424117363 (26h53m18.205050731s) [noise=true , address=77.90.31.15:4699     , piece_id=474CUNSVTINCCZZCXPBET373D7PKQYHRGN32JEJZYKJ3X6ZJM2HQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.427234519 (26h53m18.208167896s) [noise=true , address=45.91.116.14:4189    , piece_id=DRMCWLOOWG2UFFXZA5XQP5JOJUO5WQOOUVLSAWIVF6YK24XRPTPA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.430114806 (26h53m18.211048185s) [noise=true , address=45.91.116.13:4154    , piece_id=UIIVH5FUZSCBP4ACA2426UFJ4YCEKYAI6O7BPTK7QBQYBOA34IZQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.435624120 (26h53m18.216557505s) [noise=true , address=45.91.116.12:4225    , piece_id=WJY77GRZYSH5WBHPC4IN5IYHXPTI233ZMXRZCJU7YJVP5X5OLE2Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436119655 (26h53m18.217053027s) [noise=true , address=77.90.31.21:4879     , piece_id=XQ2HLQBH7MLG5SYFTXXOAQGW2OQ4ELFWMGNQDIQQXSUNWQZ6RGWA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436405218 (26h53m18.217338582s) [noise=true , address=45.91.116.13:4155    , piece_id=AHCB324J4MEQFDIL446YZCQYXEVJLWNVPWDXDW657MADLMHN25NA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436887494 (26h53m18.217820863s) [noise=true , address=77.90.31.12:4589     , piece_id=HKSIQBOQ27NGBZURHEIYPO7NOV4TJV7WAC5N2XFF5NXG457CFF4Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.436998726 (26h53m18.217932114s) [noise=true , address=77.90.31.19:4841     , piece_id=ZYJMISQVDC24SCPRDG44XBW2KXIVZFDW36M6IICMKKRNTNQVAYSA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437527079 (26h53m18.218460459s) [noise=true , address=45.91.116.10:4083    , piece_id=PPSILB2UAR5L62JYMUHLUFRMVBFIION4ZHKOTXDCU77PJR4OOGVQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.437625274 (26h53m18.218558642s) [noise=true , address=77.90.31.18:4805     , piece_id=B227ERF7AGSHZNPPTDXGEE7QOLCWMWQMIHOQH6MQQLBPD2HZZ7WA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439469614 (26h53m18.220402995s) [noise=true , address=45.91.116.4:4940     , piece_id=YOCQOJXIQK3LRVPERFUJL6U3FP2HUHTOJEUG526LUMUOE2GQSHCQ, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:02.439557143 (26h53m18.220490506s) [noise=true , address=77.90.31.11:4553     , piece_id=FDGGPSIYSQ7LH34HNSFTE6HGZQRHRZKRVMLK7GERP2JAKBOYC3QA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: <nil>
Jul 28 00:45:32.221331076 (26h53m48.002264449s) [noise=true , address=104.36.82.16:4295    , piece_id=5RGIRJLSFZSCVHCBV5IPLBKLDY3AJWIHGUYYWPBU4OJHI7JCOV4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: write tcp 159.223.143.39:33378->104.36.82.16:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221411781 (26h53m48.002345162s) [noise=true , address=104.36.82.7:4910     , piece_id=RZWYNG5BUBMWZQXM5PORO6OGAX56Q7BKIOM5TG2NYIMOBK5EZBJA, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221457184 (26h53m48.002390547s) [noise=true , address=104.36.82.13:4187    , piece_id=ASDURYYEDLBUWLT3WMGU3X2ZVACPRQEBT6PPOAB34AHTK7R3354Q, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: rpc: dial tcp 104.36.82.13:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221473904 (26h53m48.002407294s) [noise=true , address=104.36.82.8:4969     , piece_id=GB6IZEVKFMV77V3NA2J2SMJGYOHQDFMVCFGXEQL5UNBVOMBEAY4A, seg_pos={0 0}, upload=25659]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:49546->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:45:32.221516309 (26h53m48.002449671s) [seg_pos={0 0}, upload=25659]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul 28 00:45:32.251981717 (26h53m48.032915086s) [upload=25659]: Done uploading segments...
Jul 28 00:45:32.252005327 (26h53m48.032938704s) [upload=25659]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 581.785255ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)
antiblip: 
40 out of 60 is failed (threshold=3) [...................XXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration: 
1m0.031233414s
FAIL (2025-07-28 00:46:31 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 2)
======LOGS======
0 0}, upload=25669]: Uploading piece...
Jul 28 00:45:31.631883480 (26h53m47.412816857s) [noise=true , address=104.36.82.3:4781     , piece_id=QWB2WKT3HRDYGPGL64MK554DG66O3L6FBUKBZ5NWEXDEG345M7PQ, seg_pos={0 0}, upload=25669]: Uploading piece...
Jul 28 00:45:31.632421816 (26h53m47.413355197s) [noise=true , address=104.36.82.36:4645    , piece_id=PJOXYSRFPWPUQVB6LMZASSZNN62AEDYJDYPVQRL5FCR3AS6MGIIQ, seg_pos={0 0}, upload=25669]: Uploading piece...
Jul 28 00:45:31.632437934 (26h53m47.413371308s) [noise=true , address=104.36.82.14:4255    , piece_id=POR5DQTHPYAPP7YNDJZ5D34EM5G4LDSLQHZAAKPOBKF4KTRO4KXQ, seg_pos={0 0}, upload=25669]: Uploading piece...
Jul 28 00:45:31.632845418 (26h53m47.413778801s) [noise=true , address=104.36.82.12:4181    , piece_id=QQVALI6HEC3NXEMX7GIX3MRQYZRJDN72I2AR7LN456LIDFAHQWTQ, seg_pos={0 0}, upload=25669]: Uploading piece...
Jul 28 00:45:31.632874448 (26h53m47.413807824s) [noise=true , address=104.36.82.15:4283    , piece_id=EHDU4COAQ2PYCWYZE5QIYD5KXCQBQ2MP6GVJJJAYMEYGLP5Z22SA, seg_pos={0 0}, upload=25669]: Uploading piece...
Jul 28 00:45:31.633223307 (26h53m47.414156686s) [noise=true , address=104.36.82.9:4032     , piece_id=JE64WUMIW6XFIOWQMC3DXFOCGESX6ZMOLB2KFNFXUOGJPEQKJAJQ, seg_pos={0 0}, upload=25669]: Uploading piece...
Jul 28 00:45:31.635356922 (26h53m47.416290301s) [noise=true , address=104.36.82.21:4482    , piece_id=FFXNQ5M3EIJFOZQT46XKSO3Y4BDXR25AJD4VZ7HLGY75AQBXPAKQ, seg_pos={0 0}, upload=25669]: Uploading piece...
Jul 28 00:45:31.636717919 (26h53m47.417651289s) [noise=true , address=104.36.82.10:4106    , piece_id=HK6XYKSEKSM2NNOJZF2O3UUHKJUE64HZJNOOZBXFMM5EXWDU6UDA, seg_pos={0 0}, upload=25669]: Uploading piece...
Jul 28 00:45:31.637671413 (26h53m47.418604779s) [noise=true , address=104.36.82.11:4145    , piece_id=4FFTWHLXCVFVASISXIZSSBHBFW4CI6MVYCBRM7WZ755FGRZZGKYQ, seg_pos={0 0}, upload=25669]: Uploading piece...
Jul 28 00:45:31.846616575 (26h53m47.627549958s) [upload=25669]: Next returned nil segment
Jul 28 00:45:31.846641955 (26h53m47.627575343s) [upload=25669]: Waiting for error group managing segments...
Jul 28 00:45:32.227912637 (26h53m48.008846009s) [noise=true , address=79.127.205.245:4219  , piece_id=2KB4DPZGFAX2TA7MNVRRNCFZA5WM2KHUFVRYARSE6LNONYRUIR5A, seg_pos={0 0}, upload=25669]: Done uploading piece. err: <nil>
Jul 28 00:45:33.910148260 (26h53m49.691081635s) [noise=true , address=79.127.201.215:4344  , piece_id=GD7W5MOBDCIBZJHO2TUIR3FCYSYUKH6WWXUDZZYKFZLXXZE4P2SQ, seg_pos={0 0}, upload=25669]: Done uploading piece. err: <nil>
Jul 28 00:46:31.562915424 (26h54m47.343848805s) [noise=true , address=104.36.82.11:4145    , piece_id=4FFTWHLXCVFVASISXIZSSBHBFW4CI6MVYCBRM7WZ755FGRZZGKYQ, seg_pos={0 0}, upload=25669]: Done uploading piece. err: ecclient: upload failed (node:12sTUH6Y4HBG8ocrPAuuFJiVrcAm1nhpNhJXLW4HaCMdtYmyukp, address:104.36.82.11:4145): protocol: rpc: dial tcp 104.36.82.11:4145: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:46:31.563006052 (26h54m47.34393942s) [noise=true , address=104.36.82.21:4482    , piece_id=FFXNQ5M3EIJFOZQT46XKSO3Y4BDXR25AJD4VZ7HLGY75AQBXPAKQ, seg_pos={0 0}, upload=25669]: Done uploading piece. err: ecclient: upload failed (node:12B2s428CMVK7mdLV84VgWFcdRQaVQxcA7YuqqeqEM8FAefEGt7, address:104.36.82.21:4482): protocol: write tcp 159.223.143.39:58734->104.36.82.21:4482: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:46:31.564462656 (26h54m47.345396024s) [noise=true , address=104.36.82.10:4106    , piece_id=HK6XYKSEKSM2NNOJZF2O3UUHKJUE64HZJNOOZBXFMM5EXWDU6UDA, seg_pos={0 0}, upload=25669]: Done uploading piece. err: ecclient: upload failed (node:12CsQq2BuV9hNc5fVAsw882NkDJMEXZeNwhV2FGNGKAU6wuFBHm, address:104.36.82.10:4106): protocol: context deadline exceeded; context deadline exceeded
Jul 28 00:46:31.564635017 (26h54m47.345568399s) [noise=true , address=104.36.82.53:4146    , piece_id=A26Q5LQAMJLED4EJHZUUY4SNG44VJCZR2PQPMDBKMAY5EXHZ536Q, seg_pos={0 0}, upload=25669]: Done uploading piece. err: ecclient: upload failed (node:1ecwZFchyb5HmyKwRFfLDaVGck3xDWmgrVNYadHoTLWY5Z2Qs1, address:104.36.82.53:4146): protocol: rpc: dial tcp 104.36.82.53:4146: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:46:31.564723642 (26h54m47.345657021s) [noise=true , address=104.36.82.52:4088    , piece_id=WFGASIG6MV3AI64NQKQPE6R2KRTD6XGIOGLEYM3LWSNPKAR36RWQ, seg_pos={0 0}, upload=25669]: Done uploading piece. err: ecclient: upload failed (node:1284h6cPNVXJYMkSekGQVw8b7p89tkhpmgYLxTTYUDu2J7Wadoj, address:104.36.82.52:4088): protocol: rpc: dial tcp 104.36.82.52:4088: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:46:31.564800279 (26h54m47.345733645s) [noise=true , address=104.36.82.3:4781     , piece_id=QWB2WKT3HRDYGPGL64MK554DG66O3L6FBUKBZ5NWEXDEG345M7PQ, seg_pos={0 0}, upload=25669]: Done uploading piece. err: ecclient: upload failed (node:1Y6iagayTdoSQtXrAdeBrvrck8qMNRAmLogi5jUG8bcLrBEswn, address:104.36.82.3:4781): protocol: rpc: dial tcp 104.36.82.3:4781: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:46:31.564873981 (26h54m47.345807345s) [noise=true , address=104.36.82.12:4181    , piece_id=QQVALI6HEC3NXEMX7GIX3MRQYZRJDN72I2AR7LN456LIDFAHQWTQ, seg_pos={0 0}, upload=25669]: Done uploading piece. err: ecclient: upload failed (node:1XH92Sdi4ZiRg1iic5A9Mp8RyKZsk9WfVQHLoQ8t9Vvk2Lyxwh, address:104.36.82.12:4181): protocol: rpc: dial tcp 104.36.82.12:4181: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:46:31.564960712 (26h54m47.345894092s) [noise=true , address=104.36.82.54:4180    , piece_id=HT5MI7S6YILIYYA22VJYQOH737UHWD7QHGL4KRKI227MKRWSG5IA, seg_pos={0 0}, upload=25669]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:104.36.82.54:4180): protocol: rpc: dial tcp 104.36.82.54:4180: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:46:31.565027518 (26h54m47.345960895s) [noise=true , address=104.36.82.14:4255    , piece_id=POR5DQTHPYAPP7YNDJZ5D34EM5G4LDSLQHZAAKPOBKF4KTRO4KXQ, seg_pos={0 0}, upload=25669]: Done uploading piece. err: ecclient: upload failed (node:12fTNKdc6AZbkfsKRFZwTwV6wPz9j9ZD9t2eNkHBJNLMNcGh9gR, address:104.36.82.14:4255): protocol: rpc: dial tcp 104.36.82.14:4255: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:46:31.565105924 (26h54m47.346039302s) [noise=true , address=104.36.82.41:4764    , piece_id=W4OFEAW2NTYJOAU6L2IITZVZ2RHYJ4GKYC3KIQRTNUZ2DOPGI2RA, seg_pos={0 0}, upload=25669]: Done uploading piece. err: ecclient: upload failed (node:127hafVDbun7FoG4JdcogBmRNT6rhL2QQbLDLstFnx7HCWyiC9N, address:104.36.82.41:4764): protocol: rpc: dial tcp 104.36.82.41:4764: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:46:31.565140764 (26h54m47.346074135s) [noise=true , address=104.36.82.15:4283    , piece_id=EHDU4COAQ2PYCWYZE5QIYD5KXCQBQ2MP6GVJJJAYMEYGLP5Z22SA, seg_pos={0 0}, upload=25669]: Done uploading piece. err: ecclient: upload failed (node:1ytKjZnmERiZtgJuMJhXG5KFwUQwie27odve4LPE7Pw5hR5ZaU, address:104.36.82.15:4283): protocol: rpc: dial tcp 104.36.82.15:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:46:31.565182313 (26h54m47.346115693s) [noise=true , address=104.36.82.36:4645    , piece_id=PJOXYSRFPWPUQVB6LMZASSZNN62AEDYJDYPVQRL5FCR3AS6MGIIQ, seg_pos={0 0}, upload=25669]: Done uploading piece. err: ecclient: upload failed (node:12NPZfT1KXFLcfPTjX8zLQ4gpagKYosJbvCTKu9sK7989hDjTPm, address:104.36.82.36:4645): protocol: rpc: dial tcp 104.36.82.36:4645: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:46:31.565218589 (26h54m47.346151964s) [noise=true , address=104.36.82.9:4032     , piece_id=JE64WUMIW6XFIOWQMC3DXFOCGESX6ZMOLB2KFNFXUOGJPEQKJAJQ, seg_pos={0 0}, upload=25669]: Done uploading piece. err: ecclient: upload failed (node:1ferXUYkCNRALvKCfTAfqHrNuDJZcqAx7Qx3hr7TaaMdVGzxQQ, address:104.36.82.9:4032): protocol: rpc: dial tcp 104.36.82.9:4032: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:46:31.565259586 (26h54m47.346192971s) [noise=true , address=104.36.82.52:4106    , piece_id=HG636RZVZMUVPWDIHX2ZIG6ALVVOJX6XUQIRQUK22IZ4MVTH7IPA, seg_pos={0 0}, upload=25669]: Done uploading piece. err: ecclient: upload failed (node:12E2hMuAUbJyh5F4yey1EtRLYhzr29wxs47qY3zGxmGvFNUctk2, address:104.36.82.52:4106): protocol: rpc: dial tcp 104.36.82.52:4106: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:46:31.565285325 (26h54m47.34621869s) [noise=true , address=104.36.82.47:4526    , piece_id=MG4TP7LPO5IRG6FHSH4BNTRVL4PSK35G3DAQEIW7AVCG3UALWZ4Q, seg_pos={0 0}, upload=25669]: Done uploading piece. err: ecclient: upload failed (node:1djnDo1bR9DZLEAHGJGoaxg895sQ3tQpYmPgGXpmsAxnpAtQwk, address:104.36.82.47:4526): protocol: rpc: dial tcp 104.36.82.47:4526: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:46:31.565326448 (26h54m47.346259809s) [noise=true , address=104.36.82.15:4286    , piece_id=YZHDXS6HED4CRQZEP6AT2HMQZZBHWJEWAMVTMPHURW7WKKQTB2EQ, seg_pos={0 0}, upload=25669]: Done uploading piece. err: ecclient: upload failed (node:1cYgDBE6AN1EWzUQFPpF2g1pk25JoL7HcHTsFCUUHGB8ioLEGo, address:104.36.82.15:4286): protocol: write tcp 159.223.143.39:59054->104.36.82.15:4286: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 28 00:46:31.565360584 (26h54m47.346293969s) [seg_pos={0 0}, upload=25669]: Done waiting for segment. successful: 2 optimal: 14 errs: <nil>
Jul 28 00:46:31.592849117 (26h54m47.373782484s) [upload=25669]: Done uploading segments...
Jul 28 00:46:31.592874036 (26h54m47.373807411s) [upload=25669]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 2)
Duration: 1m0.031297975s core uplink us-select-1 us1-placement-20