Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 11 out of 60 is failed (threshold=3) [.................................................XXXXXXXXXXX] duration: 1m0.027937671s FAIL (2025-07-26 21:39:30 UTC) ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 9) ======LOGS====== 2.41:4785 , piece_id=5EHJOCMAY5VB3P3VIB4H2KWRPNQEOHC4SBRD3W4FV5SCWPZUYY2Q, seg_pos={0 0}, upload=55613]: Uploading piece... Jul 26 21:38:30.847376621 (58h3m13.692178159s) [noise=true , address=77.90.31.16:4746 , piece_id=JLQKX6C6OXPERHZAALQL5AFFG4ZUOSSWYKLEVGUQC2NDEQIHGV6Q, seg_pos={0 0}, upload=55613]: Uploading piece... Jul 26 21:38:30.847608461 (58h3m13.692409997s) [noise=true , address=104.36.82.22:4517 , piece_id=VJF7U7PVNV7KGGLFZ4QYLGEPG7OV7UUCXSYYEHSVBLRGZWVQK36Q, seg_pos={0 0}, upload=55613]: Uploading piece... Jul 26 21:38:30.848550676 (58h3m13.693352211s) [noise=true , address=64.49.13.140:4515 , piece_id=6Y3JDGYBKGMMKWUSRWRWTCKB3H657WRBS5SI5I3SV7OMUML7YH4Q, seg_pos={0 0}, upload=55613]: Uploading piece... Jul 26 21:38:30.848606415 (58h3m13.69340795s) [noise=true , address=104.36.82.56:4239 , piece_id=7VDKPXZJVJ67LKR4ZY5WDFCYAY7X4J7R3U2JFTMF25TMKIHKSV3Q, seg_pos={0 0}, upload=55613]: Uploading piece... Jul 26 21:38:30.848796878 (58h3m13.693598412s) [noise=true , address=104.36.82.53:4146 , piece_id=ZEVCBKKWGPJU23OSCZOFBHEBWT6ZDKOY5NSQUTQMJ3J4IDVR72PA, seg_pos={0 0}, upload=55613]: Uploading piece... Jul 26 21:38:30.849454691 (58h3m13.694256226s) [noise=true , address=104.36.82.46:4510 , piece_id=NMISPMYGRYMRIP5UNCDK7ULYVIFFA6HDTE7ZWT3XC4XLDRR5NY5A, seg_pos={0 0}, upload=55613]: Uploading piece... Jul 26 21:38:30.849786361 (58h3m13.694587897s) [noise=true , address=79.127.205.246:4284 , piece_id=ESE3FXVC2HNAGUXMGMAZVWBJUDF6WBLH26TY2YE2CLHMSOCGCJDQ, seg_pos={0 0}, upload=55613]: Uploading piece... Jul 26 21:38:30.850193159 (58h3m13.694994692s) [noise=true , address=45.91.116.14:4222 , piece_id=OJ6U7PJYJDL5E5NNEJ6GKK7EOIUCBIMNXTOAJ5XBMHLPCDJIGYPQ, seg_pos={0 0}, upload=55613]: Uploading piece... Jul 26 21:38:30.850611097 (58h3m13.695412636s) [noise=true , address=104.36.82.11:4144 , piece_id=IKT5VFDIQZTLLM6Q5ETMCAZXADLLUQXB5VQTTZ373QOHP6YFHU5Q, seg_pos={0 0}, upload=55613]: Uploading piece... Jul 26 21:38:30.851679888 (58h3m13.69648142s) [noise=true , address=77.90.31.18:4823 , piece_id=YS2INNZGFECUNIJGICA35K62DJ65Z3QFQMQUGQ3WAMHNCGAKWMSA, seg_pos={0 0}, upload=55613]: Uploading piece... Jul 26 21:38:30.853945144 (58h3m13.698746657s) [noise=true , address=104.36.82.54:4163 , piece_id=ANAICCFHOWRQLSVXSSOY3RD43VOHS5KJP7AT7WQC5L7LCTYSAKCA, seg_pos={0 0}, upload=55613]: Uploading piece... Jul 26 21:38:30.854922500 (58h3m13.699724031s) [noise=true , address=104.36.82.5:4871 , piece_id=5LKOONKZZ56CWW73OQE4DGS6LZOPYTQJIBAU4JXZETTPPJTIPVYA, seg_pos={0 0}, upload=55613]: Uploading piece... Jul 26 21:38:30.857149398 (58h3m13.701950931s) [noise=true , address=64.49.13.170:4725 , piece_id=BDQKZVPM6JS3UG5BPZEQQ3RZPTT5UYN3JRIBHBFPP4SIBZ74PZ3A, seg_pos={0 0}, upload=55613]: Uploading piece... Jul 26 21:38:30.858017030 (58h3m13.702818555s) [noise=true , address=104.36.82.55:4227 , piece_id=T6UBHFOQBLRVJXOMAX2GL76IJSD7PWNLASJQF7SR52ADNO7NT6VQ, seg_pos={0 0}, upload=55613]: Uploading piece... Jul 26 21:38:30.858192907 (58h3m13.702994438s) [noise=true , address=79.127.205.235:4677 , piece_id=VNMIVYQ3VNNO5MQPZ7TTYVG6TSWD3G2YEGEYLM7P5NAFRSYVFDQQ, seg_pos={0 0}, upload=55613]: Uploading piece... Jul 26 21:38:31.320752075 (58h3m14.165553592s) [upload=55613]: Next returned nil segment Jul 26 21:38:31.320771410 (58h3m14.165572934s) [upload=55613]: Waiting for error group managing segments... Jul 26 21:38:31.451561171 (58h3m14.296362687s) [noise=true , address=79.127.205.235:4677 , piece_id=VNMIVYQ3VNNO5MQPZ7TTYVG6TSWD3G2YEGEYLM7P5NAFRSYVFDQQ, seg_pos={0 0}, upload=55613]: Done uploading piece. err: <nil> Jul 26 21:38:31.471216861 (58h3m14.316018385s) [noise=true , address=79.127.205.246:4284 , piece_id=ESE3FXVC2HNAGUXMGMAZVWBJUDF6WBLH26TY2YE2CLHMSOCGCJDQ, seg_pos={0 0}, upload=55613]: Done uploading piece. err: <nil> Jul 26 21:38:31.509260961 (58h3m14.354062475s) [noise=true , address=64.49.13.140:4515 , piece_id=6Y3JDGYBKGMMKWUSRWRWTCKB3H657WRBS5SI5I3SV7OMUML7YH4Q, seg_pos={0 0}, upload=55613]: Done uploading piece. err: <nil> Jul 26 21:38:32.087200328 (58h3m14.93200186s) [noise=true , address=64.49.13.170:4725 , piece_id=BDQKZVPM6JS3UG5BPZEQQ3RZPTT5UYN3JRIBHBFPP4SIBZ74PZ3A, seg_pos={0 0}, upload=55613]: Done uploading piece. err: <nil> Jul 26 21:38:32.412579306 (58h3m15.257380837s) [noise=true , address=77.90.31.16:4746 , piece_id=JLQKX6C6OXPERHZAALQL5AFFG4ZUOSSWYKLEVGUQC2NDEQIHGV6Q, seg_pos={0 0}, upload=55613]: Done uploading piece. err: <nil> Jul 26 21:38:32.688540335 (58h3m15.533341861s) [noise=true , address=77.90.31.18:4823 , piece_id=YS2INNZGFECUNIJGICA35K62DJ65Z3QFQMQUGQ3WAMHNCGAKWMSA, seg_pos={0 0}, upload=55613]: Done uploading piece. err: <nil> Jul 26 21:38:33.347505483 (58h3m16.192307007s) [noise=true , address=77.90.31.11:4585 , piece_id=F26UCICYV6VNQ26GZ4VGT23ERQORFQXWSCMJK3LCJMK4QRIUXNNA, seg_pos={0 0}, upload=55613]: Done uploading piece. err: <nil> Jul 26 21:38:34.802764967 (58h3m17.64756649s) [noise=true , address=45.91.116.14:4222 , piece_id=OJ6U7PJYJDL5E5NNEJ6GKK7EOIUCBIMNXTOAJ5XBMHLPCDJIGYPQ, seg_pos={0 0}, upload=55613]: Done uploading piece. err: <nil> Jul 26 21:39:24.723332972 (58h4m7.568134506s) [noise=true , address=104.36.82.50:4026 , piece_id=TSIWEX7MX7VWIOVG7ZWU4Y22AFVUMK3HJ7ZUQY64OKKLF4U5QQOA, seg_pos={0 0}, upload=55613]: Done uploading piece. err: <nil> Jul 26 21:39:30.773664818 (58h4m13.618466351s) [noise=true , address=104.36.82.46:4510 , piece_id=NMISPMYGRYMRIP5UNCDK7ULYVIFFA6HDTE7ZWT3XC4XLDRR5NY5A, seg_pos={0 0}, upload=55613]: Done uploading piece. err: ecclient: upload failed (node:1c6o4CCFBx5xgnmaivGFzRGqqp8qrqwyzwTrcpwDYFh71wRQSZ, address:104.36.82.46:4510): protocol: write tcp 159.223.143.39:52690->104.36.82.46:4510: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:30.773743603 (58h4m13.618545125s) [noise=true , address=104.36.82.41:4785 , piece_id=5EHJOCMAY5VB3P3VIB4H2KWRPNQEOHC4SBRD3W4FV5SCWPZUYY2Q, seg_pos={0 0}, upload=55613]: Done uploading piece. err: ecclient: upload failed (node:1gYH83ECV8wai3CZyBSU8NNJM1UsysLYHbQDwBVKZRyjsdnUwB, address:104.36.82.41:4785): protocol: write tcp 159.223.143.39:44188->104.36.82.41:4785: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:30.773789229 (58h4m13.61859075s) [noise=true , address=104.36.82.53:4146 , piece_id=ZEVCBKKWGPJU23OSCZOFBHEBWT6ZDKOY5NSQUTQMJ3J4IDVR72PA, seg_pos={0 0}, upload=55613]: Done uploading piece. err: ecclient: upload failed (node:1ecwZFchyb5HmyKwRFfLDaVGck3xDWmgrVNYadHoTLWY5Z2Qs1, address:104.36.82.53:4146): protocol: write tcp 159.223.143.39:60930->104.36.82.53:4146: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:30.773837910 (58h4m13.618639433s) [noise=true , address=104.36.82.22:4517 , piece_id=VJF7U7PVNV7KGGLFZ4QYLGEPG7OV7UUCXSYYEHSVBLRGZWVQK36Q, seg_pos={0 0}, upload=55613]: Done uploading piece. err: ecclient: upload failed (node:1wSJWmZaz4FNEiar3UGW17tJc6XvaW4erEXbpAsWnE6FvuLRzT, address:104.36.82.22:4517): protocol: write tcp 159.223.143.39:41344->104.36.82.22:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:30.774289918 (58h4m13.61909144s) [noise=true , address=104.36.82.11:4144 , piece_id=IKT5VFDIQZTLLM6Q5ETMCAZXADLLUQXB5VQTTZ373QOHP6YFHU5Q, seg_pos={0 0}, upload=55613]: Done uploading piece. err: ecclient: upload failed (node:128246NkJjT3UFiQdSqFssY6jTAV1jye3dXAjAkFnC3DUAXb235, address:104.36.82.11:4144): protocol: write tcp 159.223.143.39:54932->104.36.82.11:4144: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:30.774352656 (58h4m13.619154181s) [noise=true , address=104.36.82.55:4227 , piece_id=T6UBHFOQBLRVJXOMAX2GL76IJSD7PWNLASJQF7SR52ADNO7NT6VQ, seg_pos={0 0}, upload=55613]: Done uploading piece. err: ecclient: upload failed (node:19qGGwWT9xUYsJyRnFHyHKPFr6g4AsckxRNVTg2shfVhoQtcCz, address:104.36.82.55:4227): protocol: write tcp 159.223.143.39:58992->104.36.82.55:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:30.774407714 (58h4m13.61920924s) [noise=true , address=104.36.82.56:4239 , piece_id=7VDKPXZJVJ67LKR4ZY5WDFCYAY7X4J7R3U2JFTMF25TMKIHKSV3Q, seg_pos={0 0}, upload=55613]: Done uploading piece. err: ecclient: upload failed (node:126Feu3gWgog12YavTHZkkUKK8wq6RynV8zsWd3WUAUYjvQwBnc, address:104.36.82.56:4239): protocol: write tcp 159.223.143.39:37418->104.36.82.56:4239: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:30.774611391 (58h4m13.619412913s) [noise=true , address=104.36.82.54:4163 , piece_id=ANAICCFHOWRQLSVXSSOY3RD43VOHS5KJP7AT7WQC5L7LCTYSAKCA, seg_pos={0 0}, upload=55613]: Done uploading piece. err: ecclient: upload failed (node:12CgPQ5hkrJdqbJvGarxKjXvYYnrXQqapL5BH2rNJNvnhpRi3pw, address:104.36.82.54:4163): protocol: write tcp 159.223.143.39:39882->104.36.82.54:4163: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:30.774842624 (58h4m13.619644145s) [noise=true , address=104.36.82.5:4871 , piece_id=5LKOONKZZ56CWW73OQE4DGS6LZOPYTQJIBAU4JXZETTPPJTIPVYA, seg_pos={0 0}, upload=55613]: Done uploading piece. err: ecclient: upload failed (node:1NEVGFZtdgMqREnYBPuBbYqwGRxExenRwncwCBLDFhuwdFyWdb, address:104.36.82.5:4871): protocol: write tcp 159.223.143.39:49486->104.36.82.5:4871: use of closed network connection; context deadline exceeded Jul 26 21:39:30.774859770 (58h4m13.619661283s) [seg_pos={0 0}, upload=55613]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil> Jul 26 21:39:30.799566722 (58h4m13.644368255s) [upload=55613]: Done uploading segments... Jul 26 21:39:30.799598594 (58h4m13.64440012s) [upload=55613]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9) Duration: 1m0.02795664s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................................................X.XX.] duration: 15.379811714s FAIL (2025-07-26 21:39:20 UTC) ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== l: rpc: dial tcp 104.36.82.6:4874: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:04.071127088 (58h2m46.9159286s) [noise=true , address=104.36.82.2:4730 , piece_id=DCDACT3346A3RIH2TEVDOUPKON23MF2RVCPI7QOX6DOKK4ZNJPVQ, seg_pos={0 0}, upload=55595]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: rpc: dial tcp 104.36.82.2:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:04.071172355 (58h2m46.915973887s) [seg_pos={0 0}, upload=55595]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 21:38:04.097485648 (58h2m46.942287179s) [upload=55595]: Done uploading segments... Jul 26 21:38:04.097515880 (58h2m46.94231741s) [upload=55595]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Jul 26 21:38:04.253038561 (58h2m47.097840122s) [upload=55604]: Starting upload Jul 26 21:38:04.253119101 (58h2m47.097920632s) [upload=55604]: Done starting upload Jul 26 21:38:04.253491465 (58h2m47.098292991s) [upload=55604]: Uploading segments... Jul 26 21:38:04.253557592 (58h2m47.098359128s) [upload=55604]: Got next segment. Inline: false Jul 26 21:38:04.253602720 (58h2m47.098404255s) [upload=55604]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:38:04.253608801 (58h2m47.098410338s) [upload=55604]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:38:04.361137816 (58h2m47.205939345s) [seg_pos={0 0}, upload=55604]: Begin upload segment... Jul 26 21:38:04.361201367 (58h2m47.20600288s) [seg_pos={0 0}, upload=55604]: Done begin upload segment. Jul 26 21:38:04.361228042 (58h2m47.206029566s) [noise=true , address=45.91.116.4:4938 , piece_id=Y2CLY5IUVMOI2YFPVJHWJ42Y2XAU4W2NHEC76VSP2II27XFZ6BKQ, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.361319061 (58h2m47.206120595s) [noise=true , address=45.91.116.13:4153 , piece_id=HBLDRHKMQ5NGLQGEHJ3FDB2BSZKX7OQEOMW4BYLUUYYB5YWEW4UQ, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.361841399 (58h2m47.206642931s) [noise=true , address=45.91.116.5:4299 , piece_id=3PCIKAYO3KPV57KCJ76PAXB5A2BIK5OESLK2RWKZ2C6OR6CVT5PQ, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.361860297 (58h2m47.206661882s) [noise=true , address=45.91.116.9:4263 , piece_id=Z33MMFDV7RLFAFHEGW2DW6VMQYRCBOHNVMJTHQ3YRUGWMZ4VQEGQ, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.362207262 (58h2m47.20700879s) [noise=true , address=45.91.116.12:4225 , piece_id=IZTBG7W52VI2URXM7U7NQGCPVO2N52PK76YUNQEFJ3M5CL6ZMBCQ, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.362328935 (58h2m47.207130461s) [noise=true , address=45.91.116.6:4973 , piece_id=Q475ATN54FHHJLMVRWWP5QWBS3HYLTJ6SSYQXZKO7JRDV23CNKUA, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.362588847 (58h2m47.207390376s) [noise=true , address=104.36.82.7:4909 , piece_id=LMAXLVRYYMXOGAYQCDZX3SQZHVQTG6HAERY2QZMCBUB3CJTEYRIQ, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.362800269 (58h2m47.207601806s) [noise=true , address=104.36.82.5:4839 , piece_id=7NM4RSRRUWDR2NFDYPXD2YICAHV3A53CQMNE4YIO75OGB3W77E3Q, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.362925173 (58h2m47.207726718s) [noise=true , address=45.91.116.5:4297 , piece_id=SG4CGBIJKMTGGDNBHWMMGYLLZEYLODZR2MCCURF7U6CR3JTW2OVQ, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.363344622 (58h2m47.208146149s) [noise=true , address=77.90.31.10:4518 , piece_id=TYQFS2Z2Q3KNQS2UJTZV4KEMMWNP7HUFVQ7NPGNZGXTXJFRUC4WQ, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.363397001 (58h2m47.208198525s) [noise=true , address=45.91.116.7:4011 , piece_id=VYZGBHB6Q5GAU7EZWI5OAY5MLAW6QHXJRZVO75L7G4TCE73VVHVA, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.363775838 (58h2m47.208577374s) [noise=true , address=104.36.82.2:4730 , piece_id=EKAYL22UH5FP2RDRGJS6DNGWGANMC6OWXMSVSUJI7BSDQNAI6JDQ, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.363832710 (58h2m47.208634234s) [noise=true , address=45.91.116.7:4010 , piece_id=UFJLHO7NCDEJH2HTZWRS7EYR62ACTX5MGFZW6LJAUS47ZBLSCICQ, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.364387541 (58h2m47.209189077s) [noise=true , address=45.91.116.9:4262 , piece_id=KL46QK22IJDAORPSVTO4KZ7M2NY6OBIPZGSR64QOMJKAISYHQUCA, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.364850896 (58h2m47.209652429s) [noise=true , address=45.91.116.11:4118 , piece_id=CF2HMEFVAQV2LRJBENTIOPHQG5GPO5UDSTCQTJSOIZ52VI7PQAIA, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.365310687 (58h2m47.210112216s) [noise=true , address=104.36.82.10:4079 , piece_id=N7DAQEUB52IC5RDDMURKJNY7GHZPEPSLLPEQJNMZMDCXLTIDQ5VA, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.486260172 (58h2m47.331061701s) [upload=55604]: Next returned nil segment Jul 26 21:38:04.486281865 (58h2m47.331083399s) [upload=55604]: Waiting for error group managing segments... Jul 26 21:38:04.490565046 (58h2m47.335366573s) [noise=true , address=45.91.116.12:4225 , piece_id=IZTBG7W52VI2URXM7U7NQGCPVO2N52PK76YUNQEFJ3M5CL6ZMBCQ, seg_pos={0 0}, upload=55604]: Done uploading piece. err: <nil> Jul 26 21:38:04.492835447 (58h2m47.337636956s) [noise=true , address=45.91.116.4:4938 , piece_id=Y2CLY5IUVMOI2YFPVJHWJ42Y2XAU4W2NHEC76VSP2II27XFZ6BKQ, seg_pos={0 0}, upload=55604]: Done uploading piece. err: <nil> Jul 26 21:38:04.493455836 (58h2m47.33825736s) [noise=true , address=45.91.116.13:4153 , piece_id=HBLDRHKMQ5NGLQGEHJ3FDB2BSZKX7OQEOMW4BYLUUYYB5YWEW4UQ, seg_pos={0 0}, upload=55604]: Done uploading piece. err: <nil> Jul 26 21:38:04.494061675 (58h2m47.338863196s) [noise=true , address=45.91.116.7:4010 , piece_id=UFJLHO7NCDEJH2HTZWRS7EYR62ACTX5MGFZW6LJAUS47ZBLSCICQ, seg_pos={0 0}, upload=55604]: Done uploading piece. err: <nil> Jul 26 21:38:04.494647346 (58h2m47.339448872s) [noise=true , address=45.91.116.6:4973 , piece_id=Q475ATN54FHHJLMVRWWP5QWBS3HYLTJ6SSYQXZKO7JRDV23CNKUA, seg_pos={0 0}, upload=55604]: Done uploading piece. err: <nil> Jul 26 21:38:04.495242552 (58h2m47.340044073s) [noise=true , address=45.91.116.5:4299 , piece_id=3PCIKAYO3KPV57KCJ76PAXB5A2BIK5OESLK2RWKZ2C6OR6CVT5PQ, seg_pos={0 0}, upload=55604]: Done uploading piece. err: <nil> Jul 26 21:38:04.495932061 (58h2m47.340733571s) [noise=true , address=45.91.116.9:4263 , piece_id=Z33MMFDV7RLFAFHEGW2DW6VMQYRCBOHNVMJTHQ3YRUGWMZ4VQEGQ, seg_pos={0 0}, upload=55604]: Done uploading piece. err: <nil> Jul 26 21:38:04.496696157 (58h2m47.34149767s) [noise=true , address=45.91.116.9:4262 , piece_id=KL46QK22IJDAORPSVTO4KZ7M2NY6OBIPZGSR64QOMJKAISYHQUCA, seg_pos={0 0}, upload=55604]: Done uploading piece. err: <nil> Jul 26 21:38:04.497481590 (58h2m47.342283116s) [noise=true , address=45.91.116.11:4118 , piece_id=CF2HMEFVAQV2LRJBENTIOPHQG5GPO5UDSTCQTJSOIZ52VI7PQAIA, seg_pos={0 0}, upload=55604]: Done uploading piece. err: <nil> Jul 26 21:38:04.498063319 (58h2m47.342864847s) [noise=true , address=45.91.116.7:4011 , piece_id=VYZGBHB6Q5GAU7EZWI5OAY5MLAW6QHXJRZVO75L7G4TCE73VVHVA, seg_pos={0 0}, upload=55604]: Done uploading piece. err: <nil> Jul 26 21:38:04.498614972 (58h2m47.343416502s) [noise=true , address=45.91.116.5:4297 , piece_id=SG4CGBIJKMTGGDNBHWMMGYLLZEYLODZR2MCCURF7U6CR3JTW2OVQ, seg_pos={0 0}, upload=55604]: Done uploading piece. err: <nil> Jul 26 21:38:06.393367299 (58h2m49.238168827s) [noise=true , address=77.90.31.10:4518 , piece_id=TYQFS2Z2Q3KNQS2UJTZV4KEMMWNP7HUFVQ7NPGNZGXTXJFRUC4WQ, seg_pos={0 0}, upload=55604]: Done uploading piece. err: <nil> Jul 26 21:38:34.221526535 (58h3m17.066328071s) [noise=true , address=104.36.82.7:4909 , piece_id=LMAXLVRYYMXOGAYQCDZX3SQZHVQTG6HAERY2QZMCBUB3CJTEYRIQ, seg_pos={0 0}, upload=55604]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: rpc: dial tcp 104.36.82.7:4909: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:34.221722111 (58h3m17.066523634s) [noise=true , address=104.36.82.5:4839 , piece_id=7NM4RSRRUWDR2NFDYPXD2YICAHV3A53CQMNE4YIO75OGB3W77E3Q, seg_pos={0 0}, upload=55604]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:104.36.82.5:4839): protocol: write tcp 159.223.143.39:54146->104.36.82.5:4839: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:34.221742641 (58h3m17.066544228s) [noise=true , address=104.36.82.2:4730 , piece_id=EKAYL22UH5FP2RDRGJS6DNGWGANMC6OWXMSVSUJI7BSDQNAI6JDQ, seg_pos={0 0}, upload=55604]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: rpc: dial tcp 104.36.82.2:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:34.221830554 (58h3m17.066632082s) [noise=true , address=104.36.82.10:4079 , piece_id=N7DAQEUB52IC5RDDMURKJNY7GHZPEPSLLPEQJNMZMDCXLTIDQ5VA, seg_pos={0 0}, upload=55604]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: write tcp 159.223.143.39:58764->104.36.82.10:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:34.221900476 (58h3m17.06670201s) [seg_pos={0 0}, upload=55604]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 21:38:34.251293801 (58h3m17.096095357s) [upload=55604]: Done uploading segments... Jul 26 21:38:34.251329525 (58h3m17.096131051s) [upload=55604]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 15.379846778s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................................X.......X...] duration: 4.79795894s FAIL (2025-07-26 21:39:04 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 11) Duration: 4.797992409s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................................................X.XX] duration: 30.031905892s FAIL (2025-07-26 21:38:34 UTC) ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== l: rpc: dial tcp 104.36.82.6:4874: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:04.071127088 (58h2m46.9159286s) [noise=true , address=104.36.82.2:4730 , piece_id=DCDACT3346A3RIH2TEVDOUPKON23MF2RVCPI7QOX6DOKK4ZNJPVQ, seg_pos={0 0}, upload=55595]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: rpc: dial tcp 104.36.82.2:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:04.071172355 (58h2m46.915973887s) [seg_pos={0 0}, upload=55595]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 21:38:04.097485648 (58h2m46.942287179s) [upload=55595]: Done uploading segments... Jul 26 21:38:04.097515880 (58h2m46.94231741s) [upload=55595]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Jul 26 21:38:04.253038561 (58h2m47.097840122s) [upload=55604]: Starting upload Jul 26 21:38:04.253119101 (58h2m47.097920632s) [upload=55604]: Done starting upload Jul 26 21:38:04.253491465 (58h2m47.098292991s) [upload=55604]: Uploading segments... Jul 26 21:38:04.253557592 (58h2m47.098359128s) [upload=55604]: Got next segment. Inline: false Jul 26 21:38:04.253602720 (58h2m47.098404255s) [upload=55604]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:38:04.253608801 (58h2m47.098410338s) [upload=55604]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:38:04.361137816 (58h2m47.205939345s) [seg_pos={0 0}, upload=55604]: Begin upload segment... Jul 26 21:38:04.361201367 (58h2m47.20600288s) [seg_pos={0 0}, upload=55604]: Done begin upload segment. Jul 26 21:38:04.361228042 (58h2m47.206029566s) [noise=true , address=45.91.116.4:4938 , piece_id=Y2CLY5IUVMOI2YFPVJHWJ42Y2XAU4W2NHEC76VSP2II27XFZ6BKQ, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.361319061 (58h2m47.206120595s) [noise=true , address=45.91.116.13:4153 , piece_id=HBLDRHKMQ5NGLQGEHJ3FDB2BSZKX7OQEOMW4BYLUUYYB5YWEW4UQ, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.361841399 (58h2m47.206642931s) [noise=true , address=45.91.116.5:4299 , piece_id=3PCIKAYO3KPV57KCJ76PAXB5A2BIK5OESLK2RWKZ2C6OR6CVT5PQ, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.361860297 (58h2m47.206661882s) [noise=true , address=45.91.116.9:4263 , piece_id=Z33MMFDV7RLFAFHEGW2DW6VMQYRCBOHNVMJTHQ3YRUGWMZ4VQEGQ, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.362207262 (58h2m47.20700879s) [noise=true , address=45.91.116.12:4225 , piece_id=IZTBG7W52VI2URXM7U7NQGCPVO2N52PK76YUNQEFJ3M5CL6ZMBCQ, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.362328935 (58h2m47.207130461s) [noise=true , address=45.91.116.6:4973 , piece_id=Q475ATN54FHHJLMVRWWP5QWBS3HYLTJ6SSYQXZKO7JRDV23CNKUA, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.362588847 (58h2m47.207390376s) [noise=true , address=104.36.82.7:4909 , piece_id=LMAXLVRYYMXOGAYQCDZX3SQZHVQTG6HAERY2QZMCBUB3CJTEYRIQ, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.362800269 (58h2m47.207601806s) [noise=true , address=104.36.82.5:4839 , piece_id=7NM4RSRRUWDR2NFDYPXD2YICAHV3A53CQMNE4YIO75OGB3W77E3Q, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.362925173 (58h2m47.207726718s) [noise=true , address=45.91.116.5:4297 , piece_id=SG4CGBIJKMTGGDNBHWMMGYLLZEYLODZR2MCCURF7U6CR3JTW2OVQ, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.363344622 (58h2m47.208146149s) [noise=true , address=77.90.31.10:4518 , piece_id=TYQFS2Z2Q3KNQS2UJTZV4KEMMWNP7HUFVQ7NPGNZGXTXJFRUC4WQ, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.363397001 (58h2m47.208198525s) [noise=true , address=45.91.116.7:4011 , piece_id=VYZGBHB6Q5GAU7EZWI5OAY5MLAW6QHXJRZVO75L7G4TCE73VVHVA, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.363775838 (58h2m47.208577374s) [noise=true , address=104.36.82.2:4730 , piece_id=EKAYL22UH5FP2RDRGJS6DNGWGANMC6OWXMSVSUJI7BSDQNAI6JDQ, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.363832710 (58h2m47.208634234s) [noise=true , address=45.91.116.7:4010 , piece_id=UFJLHO7NCDEJH2HTZWRS7EYR62ACTX5MGFZW6LJAUS47ZBLSCICQ, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.364387541 (58h2m47.209189077s) [noise=true , address=45.91.116.9:4262 , piece_id=KL46QK22IJDAORPSVTO4KZ7M2NY6OBIPZGSR64QOMJKAISYHQUCA, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.364850896 (58h2m47.209652429s) [noise=true , address=45.91.116.11:4118 , piece_id=CF2HMEFVAQV2LRJBENTIOPHQG5GPO5UDSTCQTJSOIZ52VI7PQAIA, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.365310687 (58h2m47.210112216s) [noise=true , address=104.36.82.10:4079 , piece_id=N7DAQEUB52IC5RDDMURKJNY7GHZPEPSLLPEQJNMZMDCXLTIDQ5VA, seg_pos={0 0}, upload=55604]: Uploading piece... Jul 26 21:38:04.486260172 (58h2m47.331061701s) [upload=55604]: Next returned nil segment Jul 26 21:38:04.486281865 (58h2m47.331083399s) [upload=55604]: Waiting for error group managing segments... Jul 26 21:38:04.490565046 (58h2m47.335366573s) [noise=true , address=45.91.116.12:4225 , piece_id=IZTBG7W52VI2URXM7U7NQGCPVO2N52PK76YUNQEFJ3M5CL6ZMBCQ, seg_pos={0 0}, upload=55604]: Done uploading piece. err: <nil> Jul 26 21:38:04.492835447 (58h2m47.337636956s) [noise=true , address=45.91.116.4:4938 , piece_id=Y2CLY5IUVMOI2YFPVJHWJ42Y2XAU4W2NHEC76VSP2II27XFZ6BKQ, seg_pos={0 0}, upload=55604]: Done uploading piece. err: <nil> Jul 26 21:38:04.493455836 (58h2m47.33825736s) [noise=true , address=45.91.116.13:4153 , piece_id=HBLDRHKMQ5NGLQGEHJ3FDB2BSZKX7OQEOMW4BYLUUYYB5YWEW4UQ, seg_pos={0 0}, upload=55604]: Done uploading piece. err: <nil> Jul 26 21:38:04.494061675 (58h2m47.338863196s) [noise=true , address=45.91.116.7:4010 , piece_id=UFJLHO7NCDEJH2HTZWRS7EYR62ACTX5MGFZW6LJAUS47ZBLSCICQ, seg_pos={0 0}, upload=55604]: Done uploading piece. err: <nil> Jul 26 21:38:04.494647346 (58h2m47.339448872s) [noise=true , address=45.91.116.6:4973 , piece_id=Q475ATN54FHHJLMVRWWP5QWBS3HYLTJ6SSYQXZKO7JRDV23CNKUA, seg_pos={0 0}, upload=55604]: Done uploading piece. err: <nil> Jul 26 21:38:04.495242552 (58h2m47.340044073s) [noise=true , address=45.91.116.5:4299 , piece_id=3PCIKAYO3KPV57KCJ76PAXB5A2BIK5OESLK2RWKZ2C6OR6CVT5PQ, seg_pos={0 0}, upload=55604]: Done uploading piece. err: <nil> Jul 26 21:38:04.495932061 (58h2m47.340733571s) [noise=true , address=45.91.116.9:4263 , piece_id=Z33MMFDV7RLFAFHEGW2DW6VMQYRCBOHNVMJTHQ3YRUGWMZ4VQEGQ, seg_pos={0 0}, upload=55604]: Done uploading piece. err: <nil> Jul 26 21:38:04.496696157 (58h2m47.34149767s) [noise=true , address=45.91.116.9:4262 , piece_id=KL46QK22IJDAORPSVTO4KZ7M2NY6OBIPZGSR64QOMJKAISYHQUCA, seg_pos={0 0}, upload=55604]: Done uploading piece. err: <nil> Jul 26 21:38:04.497481590 (58h2m47.342283116s) [noise=true , address=45.91.116.11:4118 , piece_id=CF2HMEFVAQV2LRJBENTIOPHQG5GPO5UDSTCQTJSOIZ52VI7PQAIA, seg_pos={0 0}, upload=55604]: Done uploading piece. err: <nil> Jul 26 21:38:04.498063319 (58h2m47.342864847s) [noise=true , address=45.91.116.7:4011 , piece_id=VYZGBHB6Q5GAU7EZWI5OAY5MLAW6QHXJRZVO75L7G4TCE73VVHVA, seg_pos={0 0}, upload=55604]: Done uploading piece. err: <nil> Jul 26 21:38:04.498614972 (58h2m47.343416502s) [noise=true , address=45.91.116.5:4297 , piece_id=SG4CGBIJKMTGGDNBHWMMGYLLZEYLODZR2MCCURF7U6CR3JTW2OVQ, seg_pos={0 0}, upload=55604]: Done uploading piece. err: <nil> Jul 26 21:38:06.393367299 (58h2m49.238168827s) [noise=true , address=77.90.31.10:4518 , piece_id=TYQFS2Z2Q3KNQS2UJTZV4KEMMWNP7HUFVQ7NPGNZGXTXJFRUC4WQ, seg_pos={0 0}, upload=55604]: Done uploading piece. err: <nil> Jul 26 21:38:34.221526535 (58h3m17.066328071s) [noise=true , address=104.36.82.7:4909 , piece_id=LMAXLVRYYMXOGAYQCDZX3SQZHVQTG6HAERY2QZMCBUB3CJTEYRIQ, seg_pos={0 0}, upload=55604]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: rpc: dial tcp 104.36.82.7:4909: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:34.221722111 (58h3m17.066523634s) [noise=true , address=104.36.82.5:4839 , piece_id=7NM4RSRRUWDR2NFDYPXD2YICAHV3A53CQMNE4YIO75OGB3W77E3Q, seg_pos={0 0}, upload=55604]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:104.36.82.5:4839): protocol: write tcp 159.223.143.39:54146->104.36.82.5:4839: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:34.221742641 (58h3m17.066544228s) [noise=true , address=104.36.82.2:4730 , piece_id=EKAYL22UH5FP2RDRGJS6DNGWGANMC6OWXMSVSUJI7BSDQNAI6JDQ, seg_pos={0 0}, upload=55604]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: rpc: dial tcp 104.36.82.2:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:34.221830554 (58h3m17.066632082s) [noise=true , address=104.36.82.10:4079 , piece_id=N7DAQEUB52IC5RDDMURKJNY7GHZPEPSLLPEQJNMZMDCXLTIDQ5VA, seg_pos={0 0}, upload=55604]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: write tcp 159.223.143.39:58764->104.36.82.10:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:34.221900476 (58h3m17.06670201s) [seg_pos={0 0}, upload=55604]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 21:38:34.251293801 (58h3m17.096095357s) [upload=55604]: Done uploading segments... Jul 26 21:38:34.251329525 (58h3m17.096131051s) [upload=55604]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.031933686s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 10 out of 60 is failed (threshold=3) [..................................................XXXXXXXXXX] duration: 1m0.034063393s FAIL (2025-07-26 21:38:30 UTC) ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 8) ======LOGS====== ading piece... Jul 26 21:37:30.719871589 (58h2m13.564673118s) [noise=true , address=79.127.205.235:4690 , piece_id=F2IGLQN64BZVIJQ2XDIHT6VXZZHFL3ENLKTBTXAVWYTQ3SGTVZJQ, seg_pos={0 0}, upload=55594]: Uploading piece... Jul 26 21:37:30.719888692 (58h2m13.564690227s) [noise=true , address=104.36.82.22:4521 , piece_id=4OVCYNJVU5Q6UEK32NLU6PG77KA2N3HIJK77M2OTL2NXV5F7CJFQ, seg_pos={0 0}, upload=55594]: Uploading piece... Jul 26 21:37:30.720513096 (58h2m13.565314636s) [noise=true , address=79.127.201.215:4359 , piece_id=ICJP2WXUUQA6Q2CQKWKSFUPSKZRPHRU5OYCFXTIJGV7TOG5BDTOQ, seg_pos={0 0}, upload=55594]: Uploading piece... Jul 26 21:37:30.720772310 (58h2m13.565573846s) [noise=true , address=104.36.82.55:4203 , piece_id=N537VS3NPIYIUT3HQ3V6XYB5U2KIUYQBE5YNK47K247GD6OJPCYQ, seg_pos={0 0}, upload=55594]: Uploading piece... Jul 26 21:37:30.721313001 (58h2m13.566114533s) [noise=true , address=104.36.82.47:4522 , piece_id=4JLEV6DVQHOKX2HEGUKJLVB4ITIBOLNC6ELRHTKV4AQMGCDNE2DA, seg_pos={0 0}, upload=55594]: Uploading piece... Jul 26 21:37:30.721829417 (58h2m13.566630952s) [noise=true , address=79.127.220.101:4048 , piece_id=ZWRLOCVD7P6JLMO2YE6VOULGYWKYXZ735TNQ5REIHW3TOHAMYNBQ, seg_pos={0 0}, upload=55594]: Uploading piece... Jul 26 21:37:30.721979395 (58h2m13.566780934s) [noise=true , address=79.127.219.38:4009 , piece_id=AGZOV7HBWEYK63PK6KQCP6SNSWL7NB4EDXTK53EJWB3GL2BBBV2A, seg_pos={0 0}, upload=55594]: Uploading piece... Jul 26 21:37:30.723110789 (58h2m13.567912318s) [noise=true , address=77.90.31.10:4531 , piece_id=KHVKCBJUBLJ6WKUICNIRG5V2QAVSQGP65G7SGZ5JPO6WAEMUICLA, seg_pos={0 0}, upload=55594]: Uploading piece... Jul 26 21:37:30.723147146 (58h2m13.567948683s) [noise=true , address=104.36.82.54:4161 , piece_id=KM5XVCC53MBTULB4JVKVNTTWSI2X3M7O5A5ICELY2BCW45JG5DVA, seg_pos={0 0}, upload=55594]: Uploading piece... Jul 26 21:37:30.723991810 (58h2m13.568793333s) [noise=true , address=104.36.82.8:4990 , piece_id=WD7P5KRVE3E5JHJZTM2ARBAM3BWSFTH6GWIWA4QPBQ7WKR7MDWUQ, seg_pos={0 0}, upload=55594]: Uploading piece... Jul 26 21:37:30.724531271 (58h2m13.569332803s) [noise=true , address=79.127.205.234:4618 , piece_id=YDNPEUDZVVWNAHKS67ISRM7KERAAB77GX6PKCLG75WR4WQAOR7NA, seg_pos={0 0}, upload=55594]: Uploading piece... Jul 26 21:37:30.725255009 (58h2m13.570056542s) [noise=true , address=104.36.82.52:4095 , piece_id=CERHXCGTS2UQHKSVKK3RYOVSKSAY5JQBQ5OZU7O777UIOSOF3UUQ, seg_pos={0 0}, upload=55594]: Uploading piece... Jul 26 21:37:30.725613647 (58h2m13.570415179s) [noise=true , address=104.36.82.42:4803 , piece_id=QYEXKMU6PQWVLVS55ZIII2DI2ESLKPXU55PRWWK5VLEEQDI2S3NA, seg_pos={0 0}, upload=55594]: Uploading piece... Jul 26 21:37:30.726195628 (58h2m13.57099718s) [noise=true , address=79.127.205.246:4269 , piece_id=46WHPE5S42HRHTMPP4PTSPQ3EM5FWLIYIZEGTONJS4HG7J3ETRXA, seg_pos={0 0}, upload=55594]: Uploading piece... Jul 26 21:37:30.726613870 (58h2m13.571415405s) [noise=true , address=104.36.82.46:4501 , piece_id=O6KUHJWP5NWXXHGH5DKQSDZTS3T6VKD4JGPG5DADVDDIHQXO6VHA, seg_pos={0 0}, upload=55594]: Uploading piece... Jul 26 21:37:30.727250481 (58h2m13.572052015s) [noise=true , address=104.36.82.53:4135 , piece_id=WNHYXCKVCLFN6JBKU4CDBKYGKSFDJHPLS5U56A47XIYJ4Z3DLGCA, seg_pos={0 0}, upload=55594]: Uploading piece... Jul 26 21:37:30.800442129 (58h2m13.645243653s) [upload=55594]: Next returned nil segment Jul 26 21:37:30.800463764 (58h2m13.645265288s) [upload=55594]: Waiting for error group managing segments... Jul 26 21:37:30.832572537 (58h2m13.677374049s) [noise=true , address=79.127.219.38:4009 , piece_id=AGZOV7HBWEYK63PK6KQCP6SNSWL7NB4EDXTK53EJWB3GL2BBBV2A, seg_pos={0 0}, upload=55594]: Done uploading piece. err: <nil> Jul 26 21:37:30.921931596 (58h2m13.76673312s) [noise=true , address=79.127.220.101:4048 , piece_id=ZWRLOCVD7P6JLMO2YE6VOULGYWKYXZ735TNQ5REIHW3TOHAMYNBQ, seg_pos={0 0}, upload=55594]: Done uploading piece. err: <nil> Jul 26 21:37:31.328676759 (58h2m14.173478281s) [noise=true , address=79.127.205.235:4690 , piece_id=F2IGLQN64BZVIJQ2XDIHT6VXZZHFL3ENLKTBTXAVWYTQ3SGTVZJQ, seg_pos={0 0}, upload=55594]: Done uploading piece. err: <nil> Jul 26 21:37:31.350236052 (58h2m14.195037567s) [noise=true , address=79.127.205.234:4618 , piece_id=YDNPEUDZVVWNAHKS67ISRM7KERAAB77GX6PKCLG75WR4WQAOR7NA, seg_pos={0 0}, upload=55594]: Done uploading piece. err: <nil> Jul 26 21:37:31.370426623 (58h2m14.215228133s) [noise=true , address=79.127.205.246:4269 , piece_id=46WHPE5S42HRHTMPP4PTSPQ3EM5FWLIYIZEGTONJS4HG7J3ETRXA, seg_pos={0 0}, upload=55594]: Done uploading piece. err: <nil> Jul 26 21:37:31.896479022 (58h2m14.741280546s) [noise=true , address=79.127.201.215:4359 , piece_id=ICJP2WXUUQA6Q2CQKWKSFUPSKZRPHRU5OYCFXTIJGV7TOG5BDTOQ, seg_pos={0 0}, upload=55594]: Done uploading piece. err: <nil> Jul 26 21:37:32.289053574 (58h2m15.133855098s) [noise=true , address=77.90.31.10:4531 , piece_id=KHVKCBJUBLJ6WKUICNIRG5V2QAVSQGP65G7SGZ5JPO6WAEMUICLA, seg_pos={0 0}, upload=55594]: Done uploading piece. err: <nil> Jul 26 21:38:18.619089361 (58h3m1.463890873s) [noise=true , address=104.36.82.22:4521 , piece_id=4OVCYNJVU5Q6UEK32NLU6PG77KA2N3HIJK77M2OTL2NXV5F7CJFQ, seg_pos={0 0}, upload=55594]: Done uploading piece. err: <nil> Jul 26 21:38:30.651343002 (58h3m13.496144532s) [noise=true , address=104.36.82.55:4203 , piece_id=N537VS3NPIYIUT3HQ3V6XYB5U2KIUYQBE5YNK47K247GD6OJPCYQ, seg_pos={0 0}, upload=55594]: Done uploading piece. err: ecclient: upload failed (node:12ZewP4Cfpax2814Hqbtfvp4PzcDX455Gv7WwSTSwt1NTMXzYhU, address:104.36.82.55:4203): protocol: rpc: dial tcp 104.36.82.55:4203: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:30.651438683 (58h3m13.496240214s) [noise=true , address=104.36.82.47:4522 , piece_id=4JLEV6DVQHOKX2HEGUKJLVB4ITIBOLNC6ELRHTKV4AQMGCDNE2DA, seg_pos={0 0}, upload=55594]: Done uploading piece. err: ecclient: upload failed (node:1zGZe6e37uop4y622fU1GbspMPVEZURS3FbMrf7je6VT9TYGFr, address:104.36.82.47:4522): protocol: rpc: dial tcp 104.36.82.47:4522: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:30.651505168 (58h3m13.496306695s) [noise=true , address=104.36.82.52:4095 , piece_id=CERHXCGTS2UQHKSVKK3RYOVSKSAY5JQBQ5OZU7O777UIOSOF3UUQ, seg_pos={0 0}, upload=55594]: Done uploading piece. err: ecclient: upload failed (node:1idvNc8SGRP7tkAckr2TLv7LnvftiRUTGcwy5GCrH5rVyDEK5N, address:104.36.82.52:4095): protocol: rpc: dial tcp 104.36.82.52:4095: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:30.651566868 (58h3m13.496368396s) [noise=true , address=104.36.82.42:4803 , piece_id=QYEXKMU6PQWVLVS55ZIII2DI2ESLKPXU55PRWWK5VLEEQDI2S3NA, seg_pos={0 0}, upload=55594]: Done uploading piece. err: ecclient: upload failed (node:12GFk3GkhB88UKQdZM8vZ9VLNuuYDrS5ThZjVAdbL2CtTQVMDEB, address:104.36.82.42:4803): protocol: rpc: dial tcp 104.36.82.42:4803: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:30.652289001 (58h3m13.497090517s) [noise=true , address=104.36.82.53:4135 , piece_id=WNHYXCKVCLFN6JBKU4CDBKYGKSFDJHPLS5U56A47XIYJ4Z3DLGCA, seg_pos={0 0}, upload=55594]: Done uploading piece. err: ecclient: upload failed (node:128j2QoFq4UDWCXjV8uChmN9KB1sQMnhdF6hGgNQygdqHvw9tPj, address:104.36.82.53:4135): protocol: rpc: dial tcp 104.36.82.53:4135: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:30.652368991 (58h3m13.497170527s) [noise=true , address=104.36.82.54:4161 , piece_id=KM5XVCC53MBTULB4JVKVNTTWSI2X3M7O5A5ICELY2BCW45JG5DVA, seg_pos={0 0}, upload=55594]: Done uploading piece. err: ecclient: upload failed (node:12pyhMcYjL5b9posZurgKXBnCHaMhaEjL3q3YWyc3aWkcYwbvTD, address:104.36.82.54:4161): protocol: rpc: dial tcp 104.36.82.54:4161: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:30.652400946 (58h3m13.497202474s) [noise=true , address=104.36.82.56:4231 , piece_id=HEXDGAPGNVXO6X4XW6WL4AJKKG63DLENMJ6RGRAPOUEHI4T7MWVQ, seg_pos={0 0}, upload=55594]: Done uploading piece. err: ecclient: upload failed (node:129WWyyn7SrJFbFJ2kjPVhwDNRoYM7Kj449TZiZLzUbqpwK7qJ4, address:104.36.82.56:4231): protocol: rpc: dial tcp 104.36.82.56:4231: i/o timeout; conn closed: op Jul 26 21:38:30.652450805 (58h3m13.49725233s) [noise=true , address=104.36.82.8:4990 , piece_id=WD7P5KRVE3E5JHJZTM2ARBAM3BWSFTH6GWIWA4QPBQ7WKR7MDWUQ, seg_pos={0 0}, upload=55594]: Done uploading piece. err: ecclient: upload failed (node:1pXfQm637dESPkrLBWSNALryxEwooBFdQCh1oyB4emUFes2pvx, address:104.36.82.8:4990): protocol: rpc: dial tcp 104.36.82.8:4990: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:30.652548842 (58h3m13.497350368s) [noise=true , address=104.36.82.46:4501 , piece_id=O6KUHJWP5NWXXHGH5DKQSDZTS3T6VKD4JGPG5DADVDDIHQXO6VHA, seg_pos={0 0}, upload=55594]: Done uploading piece. err: ecclient: upload failed (node:1hmsSAiSp1CBFqKhVE2S3sFfqYYkGVAiV1GfrT4Rd7hBaFNPpT, address:104.36.82.46:4501): protocol: rpc: dial tcp 104.36.82.46:4501: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:30.652619678 (58h3m13.497421205s) [noise=true , address=104.36.82.49:4985 , piece_id=4LC7WCQ3YJO7PCNEH3U36UB6Q3NLIEMHWPODXRZKOSJWWE7YQ2NA, seg_pos={0 0}, upload=55594]: Done uploading piece. err: ecclient: upload failed (node:1rp73cCs69rhnqNr1dPffoWvGNezRvcpC5WAbUbaNbkbuCfUsJ, address:104.36.82.49:4985): protocol: write tcp 159.223.143.39:47298->104.36.82.49:4985: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:30.652649069 (58h3m13.497450599s) [seg_pos={0 0}, upload=55594]: Done waiting for segment. successful: 8 optimal: 14 errs: <nil> Jul 26 21:38:30.683099774 (58h3m13.527901289s) [upload=55594]: Done uploading segments... Jul 26 21:38:30.683122588 (58h3m13.527924114s) [upload=55594]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 8) Duration: 1m0.034083655s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................................X.......X..] duration: 445.261156ms FAIL (2025-07-26 21:38:18 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 11) Duration: 445.290482ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................X.X] duration: 30.032325548s FAIL (2025-07-26 21:38:04 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== EGCRSQCVYTHN5N4JNMFFTQR4KXF5UODZ3KXL66Q, seg_pos={0 0}, upload=55585]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc): protocol: write tcp 159.223.143.39:33852->104.36.82.9:4006: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:37:06.054067249 (58h1m48.898868778s) [seg_pos={0 0}, upload=55585]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 26 21:37:06.054161181 (58h1m48.898962707s) [upload=55585]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 21:37:06.054167209 (58h1m48.898968736s) [upload=55585]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 21:37:06.127702228 (58h1m48.972503752s) [upload=55585]: Done uploading segments... Jul 26 21:37:06.127721391 (58h1m48.972522905s) [upload=55585]: Upload finished. err: <nil> Jul 26 21:37:34.108815270 (58h2m16.953616803s) [upload=55595]: Starting upload Jul 26 21:37:34.108912259 (58h2m16.953713791s) [upload=55595]: Done starting upload Jul 26 21:37:34.109040144 (58h2m16.953841655s) [upload=55595]: Uploading segments... Jul 26 21:37:34.109457366 (58h2m16.954258916s) [upload=55595]: Got next segment. Inline: false Jul 26 21:37:34.109546315 (58h2m16.954347857s) [upload=55595]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:37:34.109554870 (58h2m16.954356417s) [upload=55595]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:37:34.161439841 (58h2m17.006241373s) [seg_pos={0 0}, upload=55595]: Begin upload segment... Jul 26 21:37:34.161512816 (58h2m17.006314349s) [seg_pos={0 0}, upload=55595]: Done begin upload segment. Jul 26 21:37:34.161541976 (58h2m17.006343501s) [noise=true , address=45.91.116.8:4045 , piece_id=5IWGL5QP6QAICLCMDCHAKUDIUE24WQPGJDVMH7EN333IUC7GVNJA, seg_pos={0 0}, upload=55595]: Uploading piece... Jul 26 21:37:34.162041801 (58h2m17.006843327s) [noise=true , address=45.91.116.10:4081 , piece_id=L7P5KK5ODZ2W4N6XKYEIEKWXCIYGT2BCUUBMPD2NSN7FM67CRCXQ, seg_pos={0 0}, upload=55595]: Uploading piece... Jul 26 21:37:34.162451165 (58h2m17.007252701s) [noise=true , address=77.90.31.10:4520 , piece_id=UWPS3JAE7UOULG3U44KOCWQOFT2QZXQDBGUMSYOC77BYGZIULKJA, seg_pos={0 0}, upload=55595]: Uploading piece... Jul 26 21:37:34.162845244 (58h2m17.007646777s) [noise=true , address=104.36.82.2:4732 , piece_id=P6CTHLWK5TCNKLRPLZICH2T2GMCDETOOMZH7CL3HQOA3JFP5GG3A, seg_pos={0 0}, upload=55595]: Uploading piece... Jul 26 21:37:34.163674853 (58h2m17.008476365s) [noise=true , address=45.91.116.7:4011 , piece_id=2MJVSEIPWMLY6KVT7RYRIVEXJAX46QA5MBMK2XA7AQNVSUSK66UQ, seg_pos={0 0}, upload=55595]: Uploading piece... Jul 26 21:37:34.164091019 (58h2m17.008892543s) [noise=true , address=104.36.82.6:4874 , piece_id=XDZWOJEZ64TT3JOIGHFJ7JETR66AZWDFQYLHZRVBZSR6NXRKJZEA, seg_pos={0 0}, upload=55595]: Uploading piece... Jul 26 21:37:34.164589078 (58h2m17.009390599s) [noise=true , address=104.36.82.2:4730 , piece_id=DCDACT3346A3RIH2TEVDOUPKON23MF2RVCPI7QOX6DOKK4ZNJPVQ, seg_pos={0 0}, upload=55595]: Uploading piece... Jul 26 21:37:34.164983606 (58h2m17.009785129s) [noise=true , address=104.36.82.13:4187 , piece_id=2LF7QTQTHG5BVON3FRJNC7GDVG3APVFIQ7ZQM26J2HBLVXLJTZYQ, seg_pos={0 0}, upload=55595]: Uploading piece... Jul 26 21:37:34.165345743 (58h2m17.010147275s) [noise=true , address=45.91.116.11:4117 , piece_id=57BDDJ5P77RBY7PRFWUCVTG6PVMCFLYLK73EO6ZBPYM4GLVKUH2A, seg_pos={0 0}, upload=55595]: Uploading piece... Jul 26 21:37:34.165696841 (58h2m17.010498365s) [noise=true , address=45.91.116.9:4261 , piece_id=Q4RY7H4Q6YRSMAU64L4AXZGRXXEHC3LU6UEV4WKBI6JVMXUX2F2Q, seg_pos={0 0}, upload=55595]: Uploading piece... Jul 26 21:37:34.165816946 (58h2m17.010618474s) [noise=true , address=45.91.116.13:4154 , piece_id=N6MYA7Q7ZHNAF5TSXNLLP2CV6JNGTFLGLNULFFJXBZ7LSUATMK5Q, seg_pos={0 0}, upload=55595]: Uploading piece... Jul 26 21:37:34.166179602 (58h2m17.010981125s) [noise=true , address=45.91.116.13:4153 , piece_id=AOSYO5EUKRNEWJZICJKMLLFPTOTZN6D5IMOQO5Z7CEROKEDLPTBA, seg_pos={0 0}, upload=55595]: Uploading piece... Jul 26 21:37:34.166613427 (58h2m17.011414959s) [noise=true , address=45.91.116.4:4938 , piece_id=QH52B75NCT6JJXOFJDNUEHWI23FU64TQ3HP6FVHVZ2TWPZ7RODYA, seg_pos={0 0}, upload=55595]: Uploading piece... Jul 26 21:37:34.167013269 (58h2m17.011814795s) [noise=true , address=77.90.31.21:4879 , piece_id=MOISGVQ2ZIHXTNBCECRMFEIWNLQB35CXCZS6VVCUMRRLZ6CRPBFQ, seg_pos={0 0}, upload=55595]: Uploading piece... Jul 26 21:37:34.167411507 (58h2m17.01221305s) [noise=true , address=77.90.31.12:4589 , piece_id=IO4IWQKZOJ7DUIGGFMXFCIVCREGOCAFGWZCAOY23Q5JQKPASL4WA, seg_pos={0 0}, upload=55595]: Uploading piece... Jul 26 21:37:34.168467125 (58h2m17.013268654s) [noise=true , address=45.91.116.7:4009 , piece_id=NSLFWMPL2CNPJXEG6BJD3EZ62HP4WDLOJ7QGSR3LE3J57VAWQ5EA, seg_pos={0 0}, upload=55595]: Uploading piece... Jul 26 21:37:34.265107405 (58h2m17.109908937s) [upload=55595]: Next returned nil segment Jul 26 21:37:34.265144907 (58h2m17.109946441s) [upload=55595]: Waiting for error group managing segments... Jul 26 21:37:34.269136703 (58h2m17.113938226s) [noise=true , address=45.91.116.7:4009 , piece_id=NSLFWMPL2CNPJXEG6BJD3EZ62HP4WDLOJ7QGSR3LE3J57VAWQ5EA, seg_pos={0 0}, upload=55595]: Done uploading piece. err: <nil> Jul 26 21:37:34.269630085 (58h2m17.1144316s) [noise=true , address=45.91.116.13:4153 , piece_id=AOSYO5EUKRNEWJZICJKMLLFPTOTZN6D5IMOQO5Z7CEROKEDLPTBA, seg_pos={0 0}, upload=55595]: Done uploading piece. err: <nil> Jul 26 21:37:34.269997529 (58h2m17.114799042s) [noise=true , address=45.91.116.4:4938 , piece_id=QH52B75NCT6JJXOFJDNUEHWI23FU64TQ3HP6FVHVZ2TWPZ7RODYA, seg_pos={0 0}, upload=55595]: Done uploading piece. err: <nil> Jul 26 21:37:34.270349578 (58h2m17.115151091s) [noise=true , address=45.91.116.7:4011 , piece_id=2MJVSEIPWMLY6KVT7RYRIVEXJAX46QA5MBMK2XA7AQNVSUSK66UQ, seg_pos={0 0}, upload=55595]: Done uploading piece. err: <nil> Jul 26 21:37:34.270715529 (58h2m17.115517041s) [noise=true , address=45.91.116.13:4154 , piece_id=N6MYA7Q7ZHNAF5TSXNLLP2CV6JNGTFLGLNULFFJXBZ7LSUATMK5Q, seg_pos={0 0}, upload=55595]: Done uploading piece. err: <nil> Jul 26 21:37:34.271123229 (58h2m17.115924752s) [noise=true , address=45.91.116.10:4081 , piece_id=L7P5KK5ODZ2W4N6XKYEIEKWXCIYGT2BCUUBMPD2NSN7FM67CRCXQ, seg_pos={0 0}, upload=55595]: Done uploading piece. err: <nil> Jul 26 21:37:34.271431996 (58h2m17.116233506s) [noise=true , address=45.91.116.9:4261 , piece_id=Q4RY7H4Q6YRSMAU64L4AXZGRXXEHC3LU6UEV4WKBI6JVMXUX2F2Q, seg_pos={0 0}, upload=55595]: Done uploading piece. err: <nil> Jul 26 21:37:34.271602574 (58h2m17.116404113s) [noise=true , address=45.91.116.8:4045 , piece_id=5IWGL5QP6QAICLCMDCHAKUDIUE24WQPGJDVMH7EN333IUC7GVNJA, seg_pos={0 0}, upload=55595]: Done uploading piece. err: <nil> Jul 26 21:37:34.272005898 (58h2m17.11680742s) [noise=true , address=45.91.116.11:4117 , piece_id=57BDDJ5P77RBY7PRFWUCVTG6PVMCFLYLK73EO6ZBPYM4GLVKUH2A, seg_pos={0 0}, upload=55595]: Done uploading piece. err: <nil> Jul 26 21:37:35.580318554 (58h2m18.425120076s) [noise=true , address=77.90.31.12:4589 , piece_id=IO4IWQKZOJ7DUIGGFMXFCIVCREGOCAFGWZCAOY23Q5JQKPASL4WA, seg_pos={0 0}, upload=55595]: Done uploading piece. err: <nil> Jul 26 21:37:36.381369653 (58h2m19.226171257s) [noise=true , address=77.90.31.10:4520 , piece_id=UWPS3JAE7UOULG3U44KOCWQOFT2QZXQDBGUMSYOC77BYGZIULKJA, seg_pos={0 0}, upload=55595]: Done uploading piece. err: <nil> Jul 26 21:37:36.828105190 (58h2m19.672906707s) [noise=true , address=77.90.31.21:4879 , piece_id=MOISGVQ2ZIHXTNBCECRMFEIWNLQB35CXCZS6VVCUMRRLZ6CRPBFQ, seg_pos={0 0}, upload=55595]: Done uploading piece. err: <nil> Jul 26 21:38:04.070789907 (58h2m46.91559142s) [noise=true , address=104.36.82.2:4732 , piece_id=P6CTHLWK5TCNKLRPLZICH2T2GMCDETOOMZH7CL3HQOA3JFP5GG3A, seg_pos={0 0}, upload=55595]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: write tcp 159.223.143.39:42710->104.36.82.2:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:04.071026807 (58h2m46.915828318s) [noise=true , address=104.36.82.13:4187 , piece_id=2LF7QTQTHG5BVON3FRJNC7GDVG3APVFIQ7ZQM26J2HBLVXLJTZYQ, seg_pos={0 0}, upload=55595]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: write tcp 159.223.143.39:45074->104.36.82.13:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:04.071070556 (58h2m46.915872089s) [noise=true , address=104.36.82.6:4874 , piece_id=XDZWOJEZ64TT3JOIGHFJ7JETR66AZWDFQYLHZRVBZSR6NXRKJZEA, seg_pos={0 0}, upload=55595]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: rpc: dial tcp 104.36.82.6:4874: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:04.071127088 (58h2m46.9159286s) [noise=true , address=104.36.82.2:4730 , piece_id=DCDACT3346A3RIH2TEVDOUPKON23MF2RVCPI7QOX6DOKK4ZNJPVQ, seg_pos={0 0}, upload=55595]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: rpc: dial tcp 104.36.82.2:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:04.071172355 (58h2m46.915973887s) [seg_pos={0 0}, upload=55595]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 21:38:04.097485648 (58h2m46.942287179s) [upload=55595]: Done uploading segments... Jul 26 21:38:04.097515880 (58h2m46.94231741s) [upload=55595]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.032364352s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................................X.......X.] duration: 2.746409152s FAIL (2025-07-26 21:37:59 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.746441672s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................................X.......X] duration: 30.03314765s FAIL (2025-07-26 21:37:47 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.033177486s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 9 out of 60 is failed (threshold=3) [...................................................XXXXXXXXX] duration: 1m0.032255097s FAIL (2025-07-26 21:37:30 UTC) ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== ress=104.36.82.54:4179 , piece_id=4YKKBSVVS4SHNQNQ6LF7Z7I7HEJWYW2AEVBSKB2GGMOBI7LJHZSA, seg_pos={0 0}, upload=55579]: Uploading piece... Jul 26 21:36:30.650035104 (58h1m13.494836634s) [noise=true , address=104.36.82.9:4033 , piece_id=SEBQBKBHS33RR7T6GS2LTINMSKARH5QKAWUWXBXXS7RLJV6E6XEA, seg_pos={0 0}, upload=55579]: Uploading piece... Jul 26 21:36:30.650387047 (58h1m13.495188599s) [noise=true , address=104.36.82.47:4526 , piece_id=ZLVCAOBQGEUW3H7AHALMXTHFG3WAETZO6HKTLPSSFEVJBYUO45LA, seg_pos={0 0}, upload=55579]: Uploading piece... Jul 26 21:36:30.650774895 (58h1m13.495576428s) [noise=true , address=104.36.82.56:4236 , piece_id=KACWBN2R5GKK7GC3LFITKWHEWZWXZ4S5SBPU7DJPKRTEH3KRO3KQ, seg_pos={0 0}, upload=55579]: Uploading piece... Jul 26 21:36:30.654892828 (58h1m13.499694378s) [noise=true , address=45.91.116.9:4268 , piece_id=QH7XA6DF7P2M6L3TBHRV2KHMTP3B44VNU6B57YOLP7R22O2C7BLA, seg_pos={0 0}, upload=55579]: Uploading piece... Jul 26 21:36:30.655495740 (58h1m13.500297285s) [noise=true , address=104.36.82.8:4993 , piece_id=QOCK5IHWSH67NPVCXXLEJKK2HWM2VD2JC4GQTYLXOT6PMZBYF6LA, seg_pos={0 0}, upload=55579]: Uploading piece... Jul 26 21:36:30.655926747 (58h1m13.500728271s) [noise=true , address=104.36.82.36:4634 , piece_id=LTYCO22WWVMHCT6DBVNIRSISBX2U3J3M23OVR7RXTHVDRIM7TYLA, seg_pos={0 0}, upload=55579]: Uploading piece... Jul 26 21:36:30.656339178 (58h1m13.501140713s) [noise=true , address=45.91.116.14:4206 , piece_id=NL4R5V4JFPZ3FEH4W4T34SDEEV6ATW4KFAP6VUBF56GYF7NOI3AQ, seg_pos={0 0}, upload=55579]: Uploading piece... Jul 26 21:36:30.656761086 (58h1m13.501562622s) [noise=true , address=104.36.82.22:4515 , piece_id=3RVK3BQFY6CSWYM35FAQZ6XRWBDX3I5VMGORWEZIDO46DZJSW43A, seg_pos={0 0}, upload=55579]: Uploading piece... Jul 26 21:36:30.657154184 (58h1m13.501955712s) [noise=true , address=77.90.31.16:4766 , piece_id=FKMQRFVOMVNTAOFMMNMGYGZDLWFWA47UMXYYVD3PNK553SGZROXA, seg_pos={0 0}, upload=55579]: Uploading piece... Jul 26 21:36:30.658550769 (58h1m13.503352299s) [noise=true , address=77.90.31.12:4602 , piece_id=WKBBDBC6LX5F5IE5MRZ2XLMHVVRWZ25TALEXOEIKANMQJUZVFOKA, seg_pos={0 0}, upload=55579]: Uploading piece... Jul 26 21:36:30.761151690 (58h1m13.605953217s) [upload=55579]: Next returned nil segment Jul 26 21:36:30.761173464 (58h1m13.605975s) [upload=55579]: Waiting for error group managing segments... Jul 26 21:36:30.772500787 (58h1m13.617302317s) [noise=true , address=45.91.116.14:4206 , piece_id=NL4R5V4JFPZ3FEH4W4T34SDEEV6ATW4KFAP6VUBF56GYF7NOI3AQ, seg_pos={0 0}, upload=55579]: Done uploading piece. err: <nil> Jul 26 21:36:30.783402791 (58h1m13.62820432s) [noise=true , address=45.91.116.9:4268 , piece_id=QH7XA6DF7P2M6L3TBHRV2KHMTP3B44VNU6B57YOLP7R22O2C7BLA, seg_pos={0 0}, upload=55579]: Done uploading piece. err: <nil> Jul 26 21:36:31.487658987 (58h1m14.332460511s) [noise=true , address=79.127.201.215:4354 , piece_id=WVHZS6JASHZNLEOWGLNPNQ6NRPIJAHCDBTQ5EMISNOEUQQRF4K3Q, seg_pos={0 0}, upload=55579]: Done uploading piece. err: <nil> Jul 26 21:36:32.050853645 (58h1m14.895655168s) [noise=true , address=77.90.31.12:4602 , piece_id=WKBBDBC6LX5F5IE5MRZ2XLMHVVRWZ25TALEXOEIKANMQJUZVFOKA, seg_pos={0 0}, upload=55579]: Done uploading piece. err: <nil> Jul 26 21:36:32.577731640 (58h1m15.422533173s) [noise=true , address=77.90.31.16:4766 , piece_id=FKMQRFVOMVNTAOFMMNMGYGZDLWFWA47UMXYYVD3PNK553SGZROXA, seg_pos={0 0}, upload=55579]: Done uploading piece. err: <nil> Jul 26 21:37:30.555587046 (58h2m13.400388579s) [noise=true , address=104.36.82.52:4098 , piece_id=QMGTJGCBBKGOW6ADDXDGHMSZ2CIAEJ4WEME46ZNKRJZTDSABPTNQ, seg_pos={0 0}, upload=55579]: Done uploading piece. err: ecclient: upload failed (node:1SoFNdwfh4p5tXUkpfjQaev6tdA2ZdhsXUFvZNBsy5QrJ9zdqb, address:104.36.82.52:4098): protocol: rpc: dial tcp 104.36.82.52:4098: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:37:30.556299676 (58h2m13.401101209s) [noise=true , address=104.36.82.47:4526 , piece_id=ZLVCAOBQGEUW3H7AHALMXTHFG3WAETZO6HKTLPSSFEVJBYUO45LA, seg_pos={0 0}, upload=55579]: Done uploading piece. err: ecclient: upload failed (node:1djnDo1bR9DZLEAHGJGoaxg895sQ3tQpYmPgGXpmsAxnpAtQwk, address:104.36.82.47:4526): protocol: write tcp 159.223.143.39:43756->104.36.82.47:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:37:30.556458887 (58h2m13.401260402s) [noise=true , address=104.36.82.14:4254 , piece_id=VD4YVYAHVB5UE5QZEZQZXJZ6JO5S6W3WTK65PH3LOYYKQWMS2PRA, seg_pos={0 0}, upload=55579]: Done uploading piece. err: ecclient: upload failed (node:1bH5wgoULxFcdaLErG9JUnkbSfkLxv3zYu7VYhpAkRfQV2YJ91, address:104.36.82.14:4254): protocol: rpc: dial tcp 104.36.82.14:4254: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:37:30.556508329 (58h2m13.401309844s) [noise=true , address=104.36.82.55:4214 , piece_id=FT47QJWZ5CIAEPV4HW7PXU3JWMXNSH7S4U723BQMSZQN5EUZQ66Q, seg_pos={0 0}, upload=55579]: Done uploading piece. err: ecclient: upload failed (node:12LV4vwjXQvjzxs1jtSpNXZ6VvzLXbQBCYQBn5gzb7jh2DD9qpi, address:104.36.82.55:4214): protocol: write tcp 159.223.143.39:36920->104.36.82.55:4214: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:37:30.556633762 (58h2m13.401435277s) [noise=true , address=104.36.82.9:4033 , piece_id=SEBQBKBHS33RR7T6GS2LTINMSKARH5QKAWUWXBXXS7RLJV6E6XEA, seg_pos={0 0}, upload=55579]: Done uploading piece. err: ecclient: upload failed (node:1SV3LrcQW16wLewDzfyUWFB6sUgnERNJVdhz72CJmyde95M79C, address:104.36.82.9:4033): protocol: write tcp 159.223.143.39:49468->104.36.82.9:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:37:30.556725241 (58h2m13.401526757s) [noise=true , address=104.36.82.36:4634 , piece_id=LTYCO22WWVMHCT6DBVNIRSISBX2U3J3M23OVR7RXTHVDRIM7TYLA, seg_pos={0 0}, upload=55579]: Done uploading piece. err: ecclient: upload failed (node:1kwydp3q2wTrCVadJZB7WmKGK8h2R5QL7HGpUsjTpRbcnWUSb5, address:104.36.82.36:4634): protocol: write tcp 159.223.143.39:59564->104.36.82.36:4634: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:37:30.556834505 (58h2m13.401636018s) [noise=true , address=104.36.82.41:4762 , piece_id=KIGEWPYTKV5CLZ4ZUQFAXE5TTWYYOIGHYGOIZGRQ46FBZT3BVJ4A, seg_pos={0 0}, upload=55579]: Done uploading piece. err: ecclient: upload failed (node:1No1jHEK6YFSye92GGBri48Yv2DYUiESY6Gs3Z9PENjmBynS9b, address:104.36.82.41:4762): protocol: write tcp 159.223.143.39:51382->104.36.82.41:4762: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:37:30.556901700 (58h2m13.401703214s) [noise=true , address=104.36.82.8:4993 , piece_id=QOCK5IHWSH67NPVCXXLEJKK2HWM2VD2JC4GQTYLXOT6PMZBYF6LA, seg_pos={0 0}, upload=55579]: Done uploading piece. err: ecclient: upload failed (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD, address:104.36.82.8:4993): protocol: write tcp 159.223.143.39:37088->104.36.82.8:4993: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:37:30.557177335 (58h2m13.401978864s) [noise=true , address=104.36.82.36:4636 , piece_id=YQAHURFZMN562YNMJUGFCSJDAGPOO4QMNT5LPARG7U4MNBF4FDGA, seg_pos={0 0}, upload=55579]: Done uploading piece. err: ecclient: upload failed (node:1Lx4f6jtUdcZd9mVy5DgWqZ2vd4SrpSviaT7WUedxs7PLyzfHZ, address:104.36.82.36:4636): protocol: write tcp 159.223.143.39:59988->104.36.82.36:4636: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:37:30.557240007 (58h2m13.402041517s) [noise=true , address=104.36.82.54:4179 , piece_id=4YKKBSVVS4SHNQNQ6LF7Z7I7HEJWYW2AEVBSKB2GGMOBI7LJHZSA, seg_pos={0 0}, upload=55579]: Done uploading piece. err: ecclient: upload failed (node:12gh7eXJMeRFyjBP2e9ZyTQVVE1mMoMFfSkQ9ssfxJyqG63C4gP, address:104.36.82.54:4179): protocol: write tcp 159.223.143.39:37526->104.36.82.54:4179: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:37:30.557344352 (58h2m13.402145867s) [noise=true , address=104.36.82.22:4515 , piece_id=3RVK3BQFY6CSWYM35FAQZ6XRWBDX3I5VMGORWEZIDO46DZJSW43A, seg_pos={0 0}, upload=55579]: Done uploading piece. err: ecclient: upload failed (node:1ZZmn2sRUkYuNmHY3RY7skQwAYoVYth7rJ8DhmjEpzgE4Rtmxb, address:104.36.82.22:4515): protocol: write tcp 159.223.143.39:51160->104.36.82.22:4515: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:37:30.557421639 (58h2m13.402223159s) [noise=true , address=104.36.82.56:4236 , piece_id=KACWBN2R5GKK7GC3LFITKWHEWZWXZ4S5SBPU7DJPKRTEH3KRO3KQ, seg_pos={0 0}, upload=55579]: Done uploading piece. err: ecclient: upload failed (node:1pRLRqP5JEepqEKV9R3BUUhztQXnakq8QKMNjBZ1KR5TMMAXAb, address:104.36.82.56:4236): protocol: write tcp 159.223.143.39:57100->104.36.82.56:4236: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:37:30.557510412 (58h2m13.402311944s) [noise=true , address=104.36.82.15:4278 , piece_id=T2E4QBVWPHYFAGKCNENOP26MCBPO7SJOVMRCXD722Q4SKTZEDBUA, seg_pos={0 0}, upload=55579]: Done uploading piece. err: ecclient: upload failed (node:12qpbyKnJ8qXmos7gdxd46F8QXx98LUwtKD6mryaXPBDD5WFSwu, address:104.36.82.15:4278): protocol: write tcp 159.223.143.39:33394->104.36.82.15:4278: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:37:30.557532665 (58h2m13.4023342s) [seg_pos={0 0}, upload=55579]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 26 21:37:30.583911499 (58h2m13.428713029s) [upload=55579]: Done uploading segments... Jul 26 21:37:30.583946123 (58h2m13.42874765s) [upload=55579]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 1m0.032278965s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 13.808125921s FAIL (2025-07-26 21:37:06 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 5:4258 , piece_id=3EJ5PJPTEWBJ4RC7OG7ODGSWE7GYGZSRLGW2G2EOASK6FTIVEBDQ, seg_pos={0 0}, upload=55560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: write tcp 159.223.143.39:52516->104.36.82.15:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:35:08.668329706 (57h59m51.513131222s) [seg_pos={0 0}, upload=55560]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 26 21:35:08.668408169 (57h59m51.513209682s) [upload=55560]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 21:35:08.668413370 (57h59m51.513214883s) [upload=55560]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 21:35:08.776933451 (57h59m51.621734979s) [upload=55560]: Done uploading segments... Jul 26 21:35:08.776984632 (57h59m51.621786161s) [upload=55560]: Upload finished. err: <nil> Jul 26 21:35:49.181838667 (58h0m32.02664018s) [upload=55570]: Starting upload Jul 26 21:35:49.181965882 (58h0m32.026767406s) [upload=55570]: Done starting upload Jul 26 21:35:49.182063958 (58h0m32.026865483s) [upload=55570]: Uploading segments... Jul 26 21:35:49.182419229 (58h0m32.027220743s) [upload=55570]: Got next segment. Inline: false Jul 26 21:35:49.182485625 (58h0m32.027287149s) [upload=55570]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:35:49.182492104 (58h0m32.027293628s) [upload=55570]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:35:49.277352598 (58h0m32.12215412s) [seg_pos={0 0}, upload=55570]: Begin upload segment... Jul 26 21:35:49.277412923 (58h0m32.122214456s) [seg_pos={0 0}, upload=55570]: Done begin upload segment. Jul 26 21:35:49.277447144 (58h0m32.122248676s) [noise=true , address=77.90.31.19:4841 , piece_id=754ZVJQSKPA4MRFKANFZO5WEQO5LPIS3Q5CSMUB3MALBEFSV5TDQ, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.277466653 (58h0m32.122268185s) [noise=true , address=45.91.116.8:4048 , piece_id=4GO2NDYCAMOICJYQU4ZRLI6YIX6PL25CYE2BHLDZHG2NQNCMUHPQ, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.277969342 (58h0m32.122770914s) [noise=true , address=104.36.82.13:4185 , piece_id=AD342QCJQ7QCM4BZHPTRHAHHZJLLFK7G3KKKD5EMAX5I424AIFVA, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.278046554 (58h0m32.122848091s) [noise=true , address=104.36.82.10:4078 , piece_id=A4SLE3FH3GBD7A3BTJHHTALJ3JEOCC2YDDX5GKNKVB7IYKIZSJOA, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.278413711 (58h0m32.123215243s) [noise=true , address=45.91.116.11:4118 , piece_id=7HPVRMGS4URLHVFP7BOXT3XOWDKBCVU4HJNWIZFDOSNIPIDPTWIQ, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.278596012 (58h0m32.123397545s) [noise=true , address=104.36.82.5:4837 , piece_id=BAUVFPTRCJY27YTS4LTNQMUJRDEABR7G4ECKNPHCK7E5E3RUOAEQ, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.278808530 (58h0m32.123610062s) [noise=true , address=45.91.116.4:4939 , piece_id=PKFXWRQPCTDUMP5FSN2RVST2CRMRN6LUTGV77OND2CXQW6ODURNQ, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.279236003 (58h0m32.124037536s) [noise=true , address=45.91.116.13:4153 , piece_id=AEWEIKZ65FABOU3HCM3MH63DPSPL6TOG6DIYUKYMZY3PG6WO4KXA, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.281170453 (58h0m32.125971977s) [noise=true , address=45.91.116.6:4974 , piece_id=YTBDMO5F44R7NM6H2ALZARVIHAMZ53UJW5CTXVQZXGAMJVGCORYQ, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.282798008 (58h0m32.127599535s) [noise=true , address=104.36.82.16:4294 , piece_id=LYVLA3SCYLHUUQ4YUDIN55PAGOD3WUZUFNMCHW7BD2BQRB6C4YNA, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.283329995 (58h0m32.128131522s) [noise=true , address=45.91.116.5:4299 , piece_id=YXKNKTBM23HBTYUHTUJY67WH47KCTKX5PS5GBYUTXWACFOLGUMQQ, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.283804751 (58h0m32.128606281s) [noise=true , address=45.91.116.7:4009 , piece_id=ATBDWOER5N5E5GF2446QDRBTMGSAPU334THMPALZ62PMDXFWTY6A, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.284157681 (58h0m32.128959218s) [noise=true , address=104.36.82.11:4114 , piece_id=6EX5MQSAJFB25FXDGNRS27KVAEBZQYBN4DTSIC4MGVTX3B45FQLA, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.284554025 (58h0m32.129355555s) [noise=true , address=45.91.116.11:4117 , piece_id=TNVQGFETJ3FYCXGCDUUIIN6PWNQX2HGLZWXSP2KDVYEZ2NCA4RSA, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.284950046 (58h0m32.129751581s) [noise=true , address=45.91.116.14:4191 , piece_id=3LI4V3KBCV2KD3TVKW6VE4X2SWVEX45WRI6YM5HZCJUJ76LFLGAQ, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.285357935 (58h0m32.130159468s) [noise=true , address=45.91.116.9:4261 , piece_id=VIZ6ZHT5OID5XMR3CIUS5Q2YZRV7ZGEDXIE7JA75JBLCZKLF3XZQ, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.392068198 (58h0m32.236869724s) [upload=55570]: Next returned nil segment Jul 26 21:35:49.392090520 (58h0m32.236892046s) [upload=55570]: Waiting for error group managing segments... Jul 26 21:35:49.396650236 (58h0m32.241451765s) [noise=true , address=45.91.116.7:4009 , piece_id=ATBDWOER5N5E5GF2446QDRBTMGSAPU334THMPALZ62PMDXFWTY6A, seg_pos={0 0}, upload=55570]: Done uploading piece. err: <nil> Jul 26 21:35:49.397244603 (58h0m32.242046117s) [noise=true , address=45.91.116.8:4048 , piece_id=4GO2NDYCAMOICJYQU4ZRLI6YIX6PL25CYE2BHLDZHG2NQNCMUHPQ, seg_pos={0 0}, upload=55570]: Done uploading piece. err: <nil> Jul 26 21:35:49.398831822 (58h0m32.243633335s) [noise=true , address=45.91.116.9:4261 , piece_id=VIZ6ZHT5OID5XMR3CIUS5Q2YZRV7ZGEDXIE7JA75JBLCZKLF3XZQ, seg_pos={0 0}, upload=55570]: Done uploading piece. err: <nil> Jul 26 21:35:49.399389524 (58h0m32.244191033s) [noise=true , address=45.91.116.11:4118 , piece_id=7HPVRMGS4URLHVFP7BOXT3XOWDKBCVU4HJNWIZFDOSNIPIDPTWIQ, seg_pos={0 0}, upload=55570]: Done uploading piece. err: <nil> Jul 26 21:35:49.399437214 (58h0m32.244238745s) [noise=true , address=45.91.116.11:4117 , piece_id=TNVQGFETJ3FYCXGCDUUIIN6PWNQX2HGLZWXSP2KDVYEZ2NCA4RSA, seg_pos={0 0}, upload=55570]: Done uploading piece. err: <nil> Jul 26 21:35:49.400013566 (58h0m32.244815079s) [noise=true , address=45.91.116.5:4299 , piece_id=YXKNKTBM23HBTYUHTUJY67WH47KCTKX5PS5GBYUTXWACFOLGUMQQ, seg_pos={0 0}, upload=55570]: Done uploading piece. err: <nil> Jul 26 21:35:49.400301730 (58h0m32.24510326s) [noise=true , address=45.91.116.6:4974 , piece_id=YTBDMO5F44R7NM6H2ALZARVIHAMZ53UJW5CTXVQZXGAMJVGCORYQ, seg_pos={0 0}, upload=55570]: Done uploading piece. err: <nil> Jul 26 21:35:49.400632949 (58h0m32.245434467s) [noise=true , address=45.91.116.14:4191 , piece_id=3LI4V3KBCV2KD3TVKW6VE4X2SWVEX45WRI6YM5HZCJUJ76LFLGAQ, seg_pos={0 0}, upload=55570]: Done uploading piece. err: <nil> Jul 26 21:35:49.400884222 (58h0m32.245685732s) [noise=true , address=45.91.116.4:4939 , piece_id=PKFXWRQPCTDUMP5FSN2RVST2CRMRN6LUTGV77OND2CXQW6ODURNQ, seg_pos={0 0}, upload=55570]: Done uploading piece. err: <nil> Jul 26 21:35:49.412877712 (58h0m32.257679248s) [noise=true , address=45.91.116.13:4153 , piece_id=AEWEIKZ65FABOU3HCM3MH63DPSPL6TOG6DIYUKYMZY3PG6WO4KXA, seg_pos={0 0}, upload=55570]: Done uploading piece. err: <nil> Jul 26 21:35:50.510150768 (58h0m33.354952293s) [noise=true , address=77.90.31.19:4841 , piece_id=754ZVJQSKPA4MRFKANFZO5WEQO5LPIS3Q5CSMUB3MALBEFSV5TDQ, seg_pos={0 0}, upload=55570]: Done uploading piece. err: <nil> Jul 26 21:36:08.664419245 (58h0m51.509220758s) [noise=true , address=104.36.82.10:4078 , piece_id=A4SLE3FH3GBD7A3BTJHHTALJ3JEOCC2YDDX5GKNKVB7IYKIZSJOA, seg_pos={0 0}, upload=55570]: Done uploading piece. err: <nil> Jul 26 21:36:19.121688642 (58h1m1.966490165s) [noise=true , address=104.36.82.13:4185 , piece_id=AD342QCJQ7QCM4BZHPTRHAHHZJLLFK7G3KKKD5EMAX5I424AIFVA, seg_pos={0 0}, upload=55570]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:104.36.82.13:4185): protocol: rpc: dial tcp 104.36.82.13:4185: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:36:19.121761685 (58h1m1.9665632s) [noise=true , address=104.36.82.16:4294 , piece_id=LYVLA3SCYLHUUQ4YUDIN55PAGOD3WUZUFNMCHW7BD2BQRB6C4YNA, seg_pos={0 0}, upload=55570]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:104.36.82.16:4294): protocol: rpc: dial tcp 104.36.82.16:4294: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:36:19.121829812 (58h1m1.966631338s) [noise=true , address=104.36.82.5:4837 , piece_id=BAUVFPTRCJY27YTS4LTNQMUJRDEABR7G4ECKNPHCK7E5E3RUOAEQ, seg_pos={0 0}, upload=55570]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:104.36.82.5:4837): protocol: rpc: dial tcp 104.36.82.5:4837: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:36:19.121891612 (58h1m1.966693135s) [noise=true , address=104.36.82.11:4114 , piece_id=6EX5MQSAJFB25FXDGNRS27KVAEBZQYBN4DTSIC4MGVTX3B45FQLA, seg_pos={0 0}, upload=55570]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 159.223.143.39:42080->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:36:19.121911133 (58h1m1.966712666s) [seg_pos={0 0}, upload=55570]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 21:36:19.156014716 (58h1m2.000816247s) [upload=55570]: Done uploading segments... Jul 26 21:36:19.156046999 (58h1m2.000848521s) [upload=55570]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 13.808156645s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 6.541848197s FAIL (2025-07-26 21:36:40 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.541876835s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 8 out of 60 is failed (threshold=3) [....................................................XXXXXXXX] duration: 1m0.042478397s FAIL (2025-07-26 21:36:30 UTC) ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== YWVAM6A6EKHZZNTK7V5CQSSQ, seg_pos={0 0}, upload=55568]: Uploading piece... Jul 26 21:35:30.504946007 (58h0m13.349747542s) [noise=true , address=104.36.82.56:4232 , piece_id=SD5FHMPHHBFGOE66ZVILBDVSXK4HZTXCBX6WL2CFMZGHARYQFNAA, seg_pos={0 0}, upload=55568]: Uploading piece... Jul 26 21:35:30.505535169 (58h0m13.3503367s) [noise=true , address=79.127.201.215:4359 , piece_id=XGTTX5JWFUQCXVH3CHU3B4NWPBBLY3ITIVVPELBP4ZWEY4RFMGVQ, seg_pos={0 0}, upload=55568]: Uploading piece... Jul 26 21:35:30.505860066 (58h0m13.350661592s) [noise=true , address=104.36.82.47:4534 , piece_id=I3CXBUMPSGP32FSZJ2VOF7ZCLD3RVMILUMTKYDQ26G3BM4P3U7SA, seg_pos={0 0}, upload=55568]: Uploading piece... Jul 26 21:35:30.507157638 (58h0m13.351959173s) [noise=true , address=104.36.82.54:4188 , piece_id=EETPLRRXSE2WWKRLNMM2GUH2WWSLASK7RC7TQ2QNZJM4J2DHP66A, seg_pos={0 0}, upload=55568]: Uploading piece... Jul 26 21:35:30.507696379 (58h0m13.352497912s) [noise=true , address=77.90.31.21:4887 , piece_id=GJCGPHITMOOXHHFP6TDPN65Y5VX5H5UDFJGTCPSESO63W5B7GOQA, seg_pos={0 0}, upload=55568]: Uploading piece... Jul 26 21:35:30.508110470 (58h0m13.352912006s) [noise=true , address=104.36.82.19:4412 , piece_id=4LR7NYM6TWJTEIKTEIXRFTM4KYARVDVRPAO2UXTV6LJNPUAJKYBQ, seg_pos={0 0}, upload=55568]: Uploading piece... Jul 26 21:35:30.508503662 (58h0m13.353305198s) [noise=true , address=104.36.82.42:4807 , piece_id=MJKFXBVGKPOGYR4HBLZIQN52N35VT763BJENX7TQKOSLUICSBFLQ, seg_pos={0 0}, upload=55568]: Uploading piece... Jul 26 21:35:30.508728554 (58h0m13.353530091s) [noise=true , address=104.36.82.43:4868 , piece_id=AK5PALCHXKTAAK6SPFHNMIMCCLMNTTHJ3HDA6AFQVYQ4ILZLFNHQ, seg_pos={0 0}, upload=55568]: Uploading piece... Jul 26 21:35:30.509145519 (58h0m13.353947053s) [noise=true , address=104.36.82.6:4890 , piece_id=FW5YVJBWSNM6GUPNN577CQFSJ3PLFJLRRCWV6Q5KFR3XNG2G4G6Q, seg_pos={0 0}, upload=55568]: Uploading piece... Jul 26 21:35:30.509172658 (58h0m13.353974186s) [noise=true , address=104.36.82.21:4479 , piece_id=K3F73MHLNUQ4EK3D5D2Z5H4XGLD6KD33J27HDQFCXVQVC4WKKS6Q, seg_pos={0 0}, upload=55568]: Uploading piece... Jul 26 21:35:30.509514762 (58h0m13.354316295s) [noise=true , address=104.36.82.53:4127 , piece_id=R4P4RGXRZV62EXVCRGPAOGKXMD4PO5GOMLF7JGTGAHCQ6W3S5YFA, seg_pos={0 0}, upload=55568]: Uploading piece... Jul 26 21:35:30.509677782 (58h0m13.354479321s) [noise=true , address=104.36.82.43:4862 , piece_id=YGRBVCIOD7CWOMKODAV3DHE2W4DLGBFZEBYHLUU3W5P56RHDHWIQ, seg_pos={0 0}, upload=55568]: Uploading piece... Jul 26 21:35:30.509919124 (58h0m13.354720659s) [noise=true , address=104.36.82.4:4833 , piece_id=D4B5CO5UFCIYWR6CXPVSEB27MDRFHWYUE2BTZ63XY5BL4KFZNUEQ, seg_pos={0 0}, upload=55568]: Uploading piece... Jul 26 21:35:30.510148428 (58h0m13.354949962s) [noise=true , address=79.127.205.234:4607 , piece_id=XHZHGWO4M7G3ROOMJDDCNNRRYDEEDF2GQD32IUDYDOASNOTK2GQQ, seg_pos={0 0}, upload=55568]: Uploading piece... Jul 26 21:35:30.510350637 (58h0m13.355152171s) [noise=true , address=104.36.82.55:4208 , piece_id=E4O6Z2EEDRHT2G67FL7TRQ2EWYRT6EAX3WOM7FWOUJ5LH2JELEQQ, seg_pos={0 0}, upload=55568]: Uploading piece... Jul 26 21:35:30.510604823 (58h0m13.355406356s) [noise=true , address=104.36.82.19:4402 , piece_id=5VQZND5S74XQIK6HTP6FPDL4AJSVFVKKSU7IKW3MAI3WCY3O4WCA, seg_pos={0 0}, upload=55568]: Uploading piece... Jul 26 21:35:30.510770995 (58h0m13.35557253s) [noise=true , address=77.90.31.11:4575 , piece_id=BUZX26SVILTNEYJG62HMW3IJATTRYJE6JZMNXSASC3INXKKAYCSQ, seg_pos={0 0}, upload=55568]: Uploading piece... Jul 26 21:35:30.621108217 (58h0m13.465909741s) [upload=55568]: Next returned nil segment Jul 26 21:35:30.621128758 (58h0m13.465930289s) [upload=55568]: Waiting for error group managing segments... Jul 26 21:35:30.673986125 (58h0m13.518787656s) [noise=true , address=104.36.82.6:4890 , piece_id=FW5YVJBWSNM6GUPNN577CQFSJ3PLFJLRRCWV6Q5KFR3XNG2G4G6Q, seg_pos={0 0}, upload=55568]: Done uploading piece. err: <nil> Jul 26 21:35:30.943956936 (58h0m13.788758452s) [noise=true , address=79.127.201.215:4359 , piece_id=XGTTX5JWFUQCXVH3CHU3B4NWPBBLY3ITIVVPELBP4ZWEY4RFMGVQ, seg_pos={0 0}, upload=55568]: Done uploading piece. err: <nil> Jul 26 21:35:31.134550427 (58h0m13.97935194s) [noise=true , address=79.127.205.234:4607 , piece_id=XHZHGWO4M7G3ROOMJDDCNNRRYDEEDF2GQD32IUDYDOASNOTK2GQQ, seg_pos={0 0}, upload=55568]: Done uploading piece. err: <nil> Jul 26 21:35:31.259665490 (58h0m14.104467015s) [noise=true , address=45.91.116.9:4269 , piece_id=RSAFYQJIVLTQLRT752G4Z3DQAQCZYWVAM6A6EKHZZNTK7V5CQSSQ, seg_pos={0 0}, upload=55568]: Done uploading piece. err: <nil> Jul 26 21:35:31.488767039 (58h0m14.333568563s) [noise=true , address=77.90.31.21:4887 , piece_id=GJCGPHITMOOXHHFP6TDPN65Y5VX5H5UDFJGTCPSESO63W5B7GOQA, seg_pos={0 0}, upload=55568]: Done uploading piece. err: <nil> Jul 26 21:35:31.539228280 (58h0m14.384029814s) [noise=true , address=77.90.31.11:4575 , piece_id=BUZX26SVILTNEYJG62HMW3IJATTRYJE6JZMNXSASC3INXKKAYCSQ, seg_pos={0 0}, upload=55568]: Done uploading piece. err: <nil> Jul 26 21:35:39.937003679 (58h0m22.781805191s) [noise=true , address=104.36.82.19:4412 , piece_id=4LR7NYM6TWJTEIKTEIXRFTM4KYARVDVRPAO2UXTV6LJNPUAJKYBQ, seg_pos={0 0}, upload=55568]: Done uploading piece. err: <nil> Jul 26 21:35:39.957663489 (58h0m22.802465021s) [noise=true , address=104.36.82.19:4402 , piece_id=5VQZND5S74XQIK6HTP6FPDL4AJSVFVKKSU7IKW3MAI3WCY3O4WCA, seg_pos={0 0}, upload=55568]: Done uploading piece. err: <nil> Jul 26 21:35:56.060524021 (58h0m38.905325532s) [noise=true , address=104.36.82.42:4807 , piece_id=MJKFXBVGKPOGYR4HBLZIQN52N35VT763BJENX7TQKOSLUICSBFLQ, seg_pos={0 0}, upload=55568]: Done uploading piece. err: <nil> Jul 26 21:35:59.219488062 (58h0m42.064289591s) [noise=true , address=104.36.82.4:4833 , piece_id=D4B5CO5UFCIYWR6CXPVSEB27MDRFHWYUE2BTZ63XY5BL4KFZNUEQ, seg_pos={0 0}, upload=55568]: Done uploading piece. err: <nil> Jul 26 21:36:30.429923123 (58h1m13.27472465s) [noise=true , address=104.36.82.43:4862 , piece_id=YGRBVCIOD7CWOMKODAV3DHE2W4DLGBFZEBYHLUU3W5P56RHDHWIQ, seg_pos={0 0}, upload=55568]: Done uploading piece. err: ecclient: upload failed (node:1RBz5y29KovULcdWuaccKtWRJb2mVbifY6zqZAd3qzvBEffaVS, address:104.36.82.43:4862): protocol: write tcp 159.223.143.39:53280->104.36.82.43:4862: use of closed network connection; write tcp 159.223.143.39:53280->104.36.82.43:4862: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:53280->104.36.82.43:4862: use of closed network connection Jul 26 21:36:30.430021063 (58h1m13.274822588s) [noise=true , address=104.36.82.56:4232 , piece_id=SD5FHMPHHBFGOE66ZVILBDVSXK4HZTXCBX6WL2CFMZGHARYQFNAA, seg_pos={0 0}, upload=55568]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:104.36.82.56:4232): protocol: rpc: dial tcp 104.36.82.56:4232: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:36:30.430079454 (58h1m13.274880979s) [noise=true , address=104.36.82.43:4868 , piece_id=AK5PALCHXKTAAK6SPFHNMIMCCLMNTTHJ3HDA6AFQVYQ4ILZLFNHQ, seg_pos={0 0}, upload=55568]: Done uploading piece. err: ecclient: upload failed (node:124LJtYshPwu1uZJwVyjrjUMk3htyVnp9ntnZWSUb7X3bNkwU6t, address:104.36.82.43:4868): protocol: rpc: dial tcp 104.36.82.43:4868: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:36:30.430153158 (58h1m13.274954684s) [noise=true , address=104.36.82.53:4127 , piece_id=R4P4RGXRZV62EXVCRGPAOGKXMD4PO5GOMLF7JGTGAHCQ6W3S5YFA, seg_pos={0 0}, upload=55568]: Done uploading piece. err: ecclient: upload failed (node:12VM12t6Dq6SyApviDYSSUqWnnzW3wTfA4o8VsvJaah4fLj9EWh, address:104.36.82.53:4127): protocol: rpc: dial tcp 104.36.82.53:4127: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:36:30.430248897 (58h1m13.275050422s) [noise=true , address=104.36.82.47:4534 , piece_id=I3CXBUMPSGP32FSZJ2VOF7ZCLD3RVMILUMTKYDQ26G3BM4P3U7SA, seg_pos={0 0}, upload=55568]: Done uploading piece. err: ecclient: upload failed (node:1VDrNGcCnaKEmG5NN4nex1JoZAgdCEfX5CdsKbxytfLfKzPbr, address:104.36.82.47:4534): protocol: rpc: dial tcp 104.36.82.47:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:36:30.430311759 (58h1m13.275113284s) [noise=true , address=104.36.82.21:4479 , piece_id=K3F73MHLNUQ4EK3D5D2Z5H4XGLD6KD33J27HDQFCXVQVC4WKKS6Q, seg_pos={0 0}, upload=55568]: Done uploading piece. err: ecclient: upload failed (node:12VCopFEoHk99GsQNQPuWvTQZRD1bpXi8HHLKXaXRdvFThTxmAk, address:104.36.82.21:4479): protocol: rpc: dial tcp 104.36.82.21:4479: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:36:30.430378617 (58h1m13.275180142s) [noise=true , address=104.36.82.54:4188 , piece_id=EETPLRRXSE2WWKRLNMM2GUH2WWSLASK7RC7TQ2QNZJM4J2DHP66A, seg_pos={0 0}, upload=55568]: Done uploading piece. err: ecclient: upload failed (node:1HQ41iFa5jBBMhjL3LArqNfVhmKzNzF4N436n4qdod6GoNhAQG, address:104.36.82.54:4188): protocol: rpc: dial tcp 104.36.82.54:4188: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:36:30.430451526 (58h1m13.275253051s) [noise=true , address=104.36.82.55:4208 , piece_id=E4O6Z2EEDRHT2G67FL7TRQ2EWYRT6EAX3WOM7FWOUJ5LH2JELEQQ, seg_pos={0 0}, upload=55568]: Done uploading piece. err: ecclient: upload failed (node:12pTsKDUmbk8xYNnjrcYyX4qxwLSfHzsGgkUGSbrBiWPARfBrfi, address:104.36.82.55:4208): protocol: rpc: dial tcp 104.36.82.55:4208: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:36:30.430481005 (58h1m13.275282539s) [seg_pos={0 0}, upload=55568]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 26 21:36:30.468713174 (58h1m13.3135147s) [upload=55568]: Done uploading segments... Jul 26 21:36:30.468746382 (58h1m13.313547909s) [upload=55568]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 1m0.042503048s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.036792374s FAIL (2025-07-26 21:36:19 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 5:4258 , piece_id=3EJ5PJPTEWBJ4RC7OG7ODGSWE7GYGZSRLGW2G2EOASK6FTIVEBDQ, seg_pos={0 0}, upload=55560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: write tcp 159.223.143.39:52516->104.36.82.15:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:35:08.668329706 (57h59m51.513131222s) [seg_pos={0 0}, upload=55560]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 26 21:35:08.668408169 (57h59m51.513209682s) [upload=55560]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 21:35:08.668413370 (57h59m51.513214883s) [upload=55560]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 21:35:08.776933451 (57h59m51.621734979s) [upload=55560]: Done uploading segments... Jul 26 21:35:08.776984632 (57h59m51.621786161s) [upload=55560]: Upload finished. err: <nil> Jul 26 21:35:49.181838667 (58h0m32.02664018s) [upload=55570]: Starting upload Jul 26 21:35:49.181965882 (58h0m32.026767406s) [upload=55570]: Done starting upload Jul 26 21:35:49.182063958 (58h0m32.026865483s) [upload=55570]: Uploading segments... Jul 26 21:35:49.182419229 (58h0m32.027220743s) [upload=55570]: Got next segment. Inline: false Jul 26 21:35:49.182485625 (58h0m32.027287149s) [upload=55570]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:35:49.182492104 (58h0m32.027293628s) [upload=55570]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:35:49.277352598 (58h0m32.12215412s) [seg_pos={0 0}, upload=55570]: Begin upload segment... Jul 26 21:35:49.277412923 (58h0m32.122214456s) [seg_pos={0 0}, upload=55570]: Done begin upload segment. Jul 26 21:35:49.277447144 (58h0m32.122248676s) [noise=true , address=77.90.31.19:4841 , piece_id=754ZVJQSKPA4MRFKANFZO5WEQO5LPIS3Q5CSMUB3MALBEFSV5TDQ, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.277466653 (58h0m32.122268185s) [noise=true , address=45.91.116.8:4048 , piece_id=4GO2NDYCAMOICJYQU4ZRLI6YIX6PL25CYE2BHLDZHG2NQNCMUHPQ, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.277969342 (58h0m32.122770914s) [noise=true , address=104.36.82.13:4185 , piece_id=AD342QCJQ7QCM4BZHPTRHAHHZJLLFK7G3KKKD5EMAX5I424AIFVA, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.278046554 (58h0m32.122848091s) [noise=true , address=104.36.82.10:4078 , piece_id=A4SLE3FH3GBD7A3BTJHHTALJ3JEOCC2YDDX5GKNKVB7IYKIZSJOA, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.278413711 (58h0m32.123215243s) [noise=true , address=45.91.116.11:4118 , piece_id=7HPVRMGS4URLHVFP7BOXT3XOWDKBCVU4HJNWIZFDOSNIPIDPTWIQ, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.278596012 (58h0m32.123397545s) [noise=true , address=104.36.82.5:4837 , piece_id=BAUVFPTRCJY27YTS4LTNQMUJRDEABR7G4ECKNPHCK7E5E3RUOAEQ, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.278808530 (58h0m32.123610062s) [noise=true , address=45.91.116.4:4939 , piece_id=PKFXWRQPCTDUMP5FSN2RVST2CRMRN6LUTGV77OND2CXQW6ODURNQ, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.279236003 (58h0m32.124037536s) [noise=true , address=45.91.116.13:4153 , piece_id=AEWEIKZ65FABOU3HCM3MH63DPSPL6TOG6DIYUKYMZY3PG6WO4KXA, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.281170453 (58h0m32.125971977s) [noise=true , address=45.91.116.6:4974 , piece_id=YTBDMO5F44R7NM6H2ALZARVIHAMZ53UJW5CTXVQZXGAMJVGCORYQ, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.282798008 (58h0m32.127599535s) [noise=true , address=104.36.82.16:4294 , piece_id=LYVLA3SCYLHUUQ4YUDIN55PAGOD3WUZUFNMCHW7BD2BQRB6C4YNA, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.283329995 (58h0m32.128131522s) [noise=true , address=45.91.116.5:4299 , piece_id=YXKNKTBM23HBTYUHTUJY67WH47KCTKX5PS5GBYUTXWACFOLGUMQQ, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.283804751 (58h0m32.128606281s) [noise=true , address=45.91.116.7:4009 , piece_id=ATBDWOER5N5E5GF2446QDRBTMGSAPU334THMPALZ62PMDXFWTY6A, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.284157681 (58h0m32.128959218s) [noise=true , address=104.36.82.11:4114 , piece_id=6EX5MQSAJFB25FXDGNRS27KVAEBZQYBN4DTSIC4MGVTX3B45FQLA, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.284554025 (58h0m32.129355555s) [noise=true , address=45.91.116.11:4117 , piece_id=TNVQGFETJ3FYCXGCDUUIIN6PWNQX2HGLZWXSP2KDVYEZ2NCA4RSA, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.284950046 (58h0m32.129751581s) [noise=true , address=45.91.116.14:4191 , piece_id=3LI4V3KBCV2KD3TVKW6VE4X2SWVEX45WRI6YM5HZCJUJ76LFLGAQ, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.285357935 (58h0m32.130159468s) [noise=true , address=45.91.116.9:4261 , piece_id=VIZ6ZHT5OID5XMR3CIUS5Q2YZRV7ZGEDXIE7JA75JBLCZKLF3XZQ, seg_pos={0 0}, upload=55570]: Uploading piece... Jul 26 21:35:49.392068198 (58h0m32.236869724s) [upload=55570]: Next returned nil segment Jul 26 21:35:49.392090520 (58h0m32.236892046s) [upload=55570]: Waiting for error group managing segments... Jul 26 21:35:49.396650236 (58h0m32.241451765s) [noise=true , address=45.91.116.7:4009 , piece_id=ATBDWOER5N5E5GF2446QDRBTMGSAPU334THMPALZ62PMDXFWTY6A, seg_pos={0 0}, upload=55570]: Done uploading piece. err: <nil> Jul 26 21:35:49.397244603 (58h0m32.242046117s) [noise=true , address=45.91.116.8:4048 , piece_id=4GO2NDYCAMOICJYQU4ZRLI6YIX6PL25CYE2BHLDZHG2NQNCMUHPQ, seg_pos={0 0}, upload=55570]: Done uploading piece. err: <nil> Jul 26 21:35:49.398831822 (58h0m32.243633335s) [noise=true , address=45.91.116.9:4261 , piece_id=VIZ6ZHT5OID5XMR3CIUS5Q2YZRV7ZGEDXIE7JA75JBLCZKLF3XZQ, seg_pos={0 0}, upload=55570]: Done uploading piece. err: <nil> Jul 26 21:35:49.399389524 (58h0m32.244191033s) [noise=true , address=45.91.116.11:4118 , piece_id=7HPVRMGS4URLHVFP7BOXT3XOWDKBCVU4HJNWIZFDOSNIPIDPTWIQ, seg_pos={0 0}, upload=55570]: Done uploading piece. err: <nil> Jul 26 21:35:49.399437214 (58h0m32.244238745s) [noise=true , address=45.91.116.11:4117 , piece_id=TNVQGFETJ3FYCXGCDUUIIN6PWNQX2HGLZWXSP2KDVYEZ2NCA4RSA, seg_pos={0 0}, upload=55570]: Done uploading piece. err: <nil> Jul 26 21:35:49.400013566 (58h0m32.244815079s) [noise=true , address=45.91.116.5:4299 , piece_id=YXKNKTBM23HBTYUHTUJY67WH47KCTKX5PS5GBYUTXWACFOLGUMQQ, seg_pos={0 0}, upload=55570]: Done uploading piece. err: <nil> Jul 26 21:35:49.400301730 (58h0m32.24510326s) [noise=true , address=45.91.116.6:4974 , piece_id=YTBDMO5F44R7NM6H2ALZARVIHAMZ53UJW5CTXVQZXGAMJVGCORYQ, seg_pos={0 0}, upload=55570]: Done uploading piece. err: <nil> Jul 26 21:35:49.400632949 (58h0m32.245434467s) [noise=true , address=45.91.116.14:4191 , piece_id=3LI4V3KBCV2KD3TVKW6VE4X2SWVEX45WRI6YM5HZCJUJ76LFLGAQ, seg_pos={0 0}, upload=55570]: Done uploading piece. err: <nil> Jul 26 21:35:49.400884222 (58h0m32.245685732s) [noise=true , address=45.91.116.4:4939 , piece_id=PKFXWRQPCTDUMP5FSN2RVST2CRMRN6LUTGV77OND2CXQW6ODURNQ, seg_pos={0 0}, upload=55570]: Done uploading piece. err: <nil> Jul 26 21:35:49.412877712 (58h0m32.257679248s) [noise=true , address=45.91.116.13:4153 , piece_id=AEWEIKZ65FABOU3HCM3MH63DPSPL6TOG6DIYUKYMZY3PG6WO4KXA, seg_pos={0 0}, upload=55570]: Done uploading piece. err: <nil> Jul 26 21:35:50.510150768 (58h0m33.354952293s) [noise=true , address=77.90.31.19:4841 , piece_id=754ZVJQSKPA4MRFKANFZO5WEQO5LPIS3Q5CSMUB3MALBEFSV5TDQ, seg_pos={0 0}, upload=55570]: Done uploading piece. err: <nil> Jul 26 21:36:08.664419245 (58h0m51.509220758s) [noise=true , address=104.36.82.10:4078 , piece_id=A4SLE3FH3GBD7A3BTJHHTALJ3JEOCC2YDDX5GKNKVB7IYKIZSJOA, seg_pos={0 0}, upload=55570]: Done uploading piece. err: <nil> Jul 26 21:36:19.121688642 (58h1m1.966490165s) [noise=true , address=104.36.82.13:4185 , piece_id=AD342QCJQ7QCM4BZHPTRHAHHZJLLFK7G3KKKD5EMAX5I424AIFVA, seg_pos={0 0}, upload=55570]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:104.36.82.13:4185): protocol: rpc: dial tcp 104.36.82.13:4185: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:36:19.121761685 (58h1m1.9665632s) [noise=true , address=104.36.82.16:4294 , piece_id=LYVLA3SCYLHUUQ4YUDIN55PAGOD3WUZUFNMCHW7BD2BQRB6C4YNA, seg_pos={0 0}, upload=55570]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:104.36.82.16:4294): protocol: rpc: dial tcp 104.36.82.16:4294: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:36:19.121829812 (58h1m1.966631338s) [noise=true , address=104.36.82.5:4837 , piece_id=BAUVFPTRCJY27YTS4LTNQMUJRDEABR7G4ECKNPHCK7E5E3RUOAEQ, seg_pos={0 0}, upload=55570]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:104.36.82.5:4837): protocol: rpc: dial tcp 104.36.82.5:4837: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:36:19.121891612 (58h1m1.966693135s) [noise=true , address=104.36.82.11:4114 , piece_id=6EX5MQSAJFB25FXDGNRS27KVAEBZQYBN4DTSIC4MGVTX3B45FQLA, seg_pos={0 0}, upload=55570]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 159.223.143.39:42080->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:36:19.121911133 (58h1m1.966712666s) [seg_pos={0 0}, upload=55570]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 21:36:19.156014716 (58h1m2.000816247s) [upload=55570]: Done uploading segments... Jul 26 21:36:19.156046999 (58h1m2.000848521s) [upload=55570]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.036913082s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [.....................................................XXXXXXX] duration: 1m0.033154563s FAIL (2025-07-26 21:35:30 UTC) ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 8) ======LOGS====== Q73TC4GWYCXMKRWB35Y7XWK5G4PE55KNW4AWRV4DVLBWZ7GN6Q, seg_pos={0 0}, upload=55551]: Uploading piece... Jul 26 21:34:30.415138405 (57h59m13.259939943s) [noise=true , address=104.36.82.20:4455 , piece_id=DZ676A6QB6RGDHN3N6REWHXLB5PRSADX22WOMHXSYHD7KTGTWFBA, seg_pos={0 0}, upload=55551]: Uploading piece... Jul 26 21:34:30.416499456 (57h59m13.261300998s) [noise=true , address=104.36.82.46:4506 , piece_id=5JHJHBQFHPBF26MTE3XOTCWZ63ZPORLVOZB7EBBXCRDR2N45AQ4A, seg_pos={0 0}, upload=55551]: Uploading piece... Jul 26 21:34:30.417101136 (57h59m13.261902676s) [noise=true , address=104.36.82.47:4523 , piece_id=UDDQOIPCHJCXWC4SHJQI2GVI5A5TQ45SQQRJHBF2QC4KJ47HK4JQ, seg_pos={0 0}, upload=55551]: Uploading piece... Jul 26 21:34:30.417684725 (57h59m13.262486267s) [noise=true , address=104.36.82.54:4164 , piece_id=F36YDHBOZ6UCGYATIMQYTPG2LUNHIFOLO5B7SZH6TZOMRWQKXGMA, seg_pos={0 0}, upload=55551]: Uploading piece... Jul 26 21:34:30.418229982 (57h59m13.263031516s) [noise=true , address=104.36.82.53:4156 , piece_id=HCNSVWEPX4ESHQIEBPLIRQWFLKXMW6V3ZWZBS7LYMLDWZX34QIKA, seg_pos={0 0}, upload=55551]: Uploading piece... Jul 26 21:34:30.419324316 (57h59m13.26412585s) [noise=true , address=104.36.82.55:4222 , piece_id=WNSTV36Y52GHJIDNVMW3FGD4H2STERQPXM5OLXLCOWQJ4PW25BSQ, seg_pos={0 0}, upload=55551]: Uploading piece... Jul 26 21:34:30.420358356 (57h59m13.265159884s) [noise=true , address=79.127.205.246:4288 , piece_id=LFI5ALSBGVZNA5T7RVHK7VBZ4NJJ3UDVJ6AZ4A56OPODHRCJI2OA, seg_pos={0 0}, upload=55551]: Uploading piece... Jul 26 21:34:30.420897698 (57h59m13.265699234s) [noise=true , address=79.127.219.51:4210 , piece_id=PQQ4ZNJ3YKETB4K6JY27QN5KHQ52WJJQD4DAUE27K75FG7OM47YQ, seg_pos={0 0}, upload=55551]: Uploading piece... Jul 26 21:34:30.421406472 (57h59m13.266208025s) [noise=true , address=79.127.205.235:4691 , piece_id=OY3VH7D4CETVNRLLFUKTOVMD3D3M5YRWEIIML4QLXKAB36QCFBNA, seg_pos={0 0}, upload=55551]: Uploading piece... Jul 26 21:34:30.421928677 (57h59m13.266730213s) [noise=true , address=104.36.82.10:4091 , piece_id=RO37R6434DHQIIT6XOUXER7OIWLGRGDDSLCRPN4JHBHCVXQB2HUQ, seg_pos={0 0}, upload=55551]: Uploading piece... Jul 26 21:34:30.422403019 (57h59m13.267204553s) [noise=true , address=79.127.201.215:4357 , piece_id=63V5PI7Y7AVJRTIGCLJJDIFCTRHOINNRTTFDPPRMVSCEJSGWMTYA, seg_pos={0 0}, upload=55551]: Uploading piece... Jul 26 21:34:30.422891154 (57h59m13.267692691s) [noise=true , address=77.90.31.21:4900 , piece_id=BGOEPKM3YY63TCJ4GHM7RDMORCSBPVNBEFQFRMFBQISJ5SS5G6EQ, seg_pos={0 0}, upload=55551]: Uploading piece... Jul 26 21:34:30.423392891 (57h59m13.268194428s) [noise=true , address=64.49.13.140:4514 , piece_id=TOAQKNGBVNER5H7MOM7EDC5AWHSM2LWTPJIEWZFFMX4JF3UUSP5Q, seg_pos={0 0}, upload=55551]: Uploading piece... Jul 26 21:34:30.424493163 (57h59m13.269294699s) [noise=true , address=104.36.82.46:4485 , piece_id=BGWJH5OF72UOTLLKPUYRZJ4LZD6HXZPFCVFUNGFLZDFBNPSAZO5Q, seg_pos={0 0}, upload=55551]: Uploading piece... Jul 26 21:34:30.564037574 (57h59m13.408839106s) [upload=55551]: Next returned nil segment Jul 26 21:34:30.564059331 (57h59m13.408860869s) [upload=55551]: Waiting for error group managing segments... Jul 26 21:34:30.749014822 (57h59m13.593816347s) [noise=true , address=79.127.201.215:4357 , piece_id=63V5PI7Y7AVJRTIGCLJJDIFCTRHOINNRTTFDPPRMVSCEJSGWMTYA, seg_pos={0 0}, upload=55551]: Done uploading piece. err: <nil> Jul 26 21:34:30.886814060 (57h59m13.731615585s) [noise=true , address=79.127.219.51:4210 , piece_id=PQQ4ZNJ3YKETB4K6JY27QN5KHQ52WJJQD4DAUE27K75FG7OM47YQ, seg_pos={0 0}, upload=55551]: Done uploading piece. err: <nil> Jul 26 21:34:30.895916054 (57h59m13.740717577s) [noise=true , address=79.127.219.38:4015 , piece_id=VGOZY5GHBZ4T5EWQRRVYWHH7WDSQIWEOS5P6UY3UEH6KIIY25BAQ, seg_pos={0 0}, upload=55551]: Done uploading piece. err: <nil> Jul 26 21:34:31.023712165 (57h59m13.86851369s) [noise=true , address=79.127.205.246:4288 , piece_id=LFI5ALSBGVZNA5T7RVHK7VBZ4NJJ3UDVJ6AZ4A56OPODHRCJI2OA, seg_pos={0 0}, upload=55551]: Done uploading piece. err: <nil> Jul 26 21:34:31.342011632 (57h59m14.186813141s) [noise=true , address=79.127.205.235:4691 , piece_id=OY3VH7D4CETVNRLLFUKTOVMD3D3M5YRWEIIML4QLXKAB36QCFBNA, seg_pos={0 0}, upload=55551]: Done uploading piece. err: <nil> Jul 26 21:34:31.422855511 (57h59m14.267657026s) [noise=true , address=77.90.31.21:4900 , piece_id=BGOEPKM3YY63TCJ4GHM7RDMORCSBPVNBEFQFRMFBQISJ5SS5G6EQ, seg_pos={0 0}, upload=55551]: Done uploading piece. err: <nil> Jul 26 21:34:31.571475678 (57h59m14.416277195s) [noise=true , address=64.49.13.140:4514 , piece_id=TOAQKNGBVNER5H7MOM7EDC5AWHSM2LWTPJIEWZFFMX4JF3UUSP5Q, seg_pos={0 0}, upload=55551]: Done uploading piece. err: <nil> Jul 26 21:34:33.121393630 (57h59m15.966195138s) [noise=true , address=64.49.13.138:4482 , piece_id=HAR3RVITMPOWKG5LCBC4WW4X4VGF3KMPETG46PLL4TAGRJPKKNVQ, seg_pos={0 0}, upload=55551]: Done uploading piece. err: <nil> Jul 26 21:35:30.329687725 (58h0m13.174489261s) [noise=true , address=104.36.82.56:4251 , piece_id=QFQ73TC4GWYCXMKRWB35Y7XWK5G4PE55KNW4AWRV4DVLBWZ7GN6Q, seg_pos={0 0}, upload=55551]: Done uploading piece. err: ecclient: upload failed (node:12vvYKd1KXrz8cYXRJSL9KXiYQtY95aB2GFPGFi9c9Nj1Raj6J4, address:104.36.82.56:4251): protocol: rpc: dial tcp 104.36.82.56:4251: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:35:30.329804914 (58h0m13.17460644s) [noise=true , address=104.36.82.20:4455 , piece_id=DZ676A6QB6RGDHN3N6REWHXLB5PRSADX22WOMHXSYHD7KTGTWFBA, seg_pos={0 0}, upload=55551]: Done uploading piece. err: ecclient: upload failed (node:1X8uFZXHVMycvk6tELmYQyWoAS2sSFmNNWhvQFApvoWXuXFcj4, address:104.36.82.20:4455): protocol: write tcp 159.223.143.39:44232->104.36.82.20:4455: use of closed network connection; write tcp 159.223.143.39:44232->104.36.82.20:4455: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:44232->104.36.82.20:4455: use of closed network connection Jul 26 21:35:30.329875921 (58h0m13.174677447s) [noise=true , address=104.36.82.55:4222 , piece_id=WNSTV36Y52GHJIDNVMW3FGD4H2STERQPXM5OLXLCOWQJ4PW25BSQ, seg_pos={0 0}, upload=55551]: Done uploading piece. err: ecclient: upload failed (node:12ZKW35KvrsoYRefbYeUFZo8bh4XZYycNKGEvTFjzibezjXgj3e, address:104.36.82.55:4222): protocol: rpc: dial tcp 104.36.82.55:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:35:30.329962946 (58h0m13.17476447s) [noise=true , address=104.36.82.46:4485 , piece_id=BGWJH5OF72UOTLLKPUYRZJ4LZD6HXZPFCVFUNGFLZDFBNPSAZO5Q, seg_pos={0 0}, upload=55551]: Done uploading piece. err: ecclient: upload failed (node:12w74nT4D4F3tttRavcoKiPzyct4CxvGQfbn6EcVypHzaexCxNJ, address:104.36.82.46:4485): protocol: read tcp 159.223.143.39:37924->104.36.82.46:4485: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:35:30.330180134 (58h0m13.17498167s) [noise=true , address=104.36.82.54:4164 , piece_id=F36YDHBOZ6UCGYATIMQYTPG2LUNHIFOLO5B7SZH6TZOMRWQKXGMA, seg_pos={0 0}, upload=55551]: Done uploading piece. err: ecclient: upload failed (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2, address:104.36.82.54:4164): protocol: rpc: dial tcp 104.36.82.54:4164: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:35:30.330350853 (58h0m13.175152386s) [noise=true , address=104.36.82.52:4113 , piece_id=P2TFZFQKUGGB3MOPP5XWLV3E5M7GO3Y6WRA2ACWVGDEWHDTKXCCA, seg_pos={0 0}, upload=55551]: Done uploading piece. err: ecclient: upload failed (node:1brZ4RtcK5J3xCPmnbsTccF9hrnj8tqhV1WyMqqrjuitgSGMou, address:104.36.82.52:4113): protocol: rpc: dial tcp 104.36.82.52:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:35:30.330477946 (58h0m13.175279471s) [noise=true , address=104.36.82.47:4523 , piece_id=UDDQOIPCHJCXWC4SHJQI2GVI5A5TQ45SQQRJHBF2QC4KJ47HK4JQ, seg_pos={0 0}, upload=55551]: Done uploading piece. err: ecclient: upload failed (node:12pADNfAT7ccugc1jKnTsk2xiw1aSEpuoKZSRosexzHowDmDCWV, address:104.36.82.47:4523): protocol: rpc: dial tcp 104.36.82.47:4523: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:35:30.330553125 (58h0m13.175354651s) [noise=true , address=104.36.82.53:4156 , piece_id=HCNSVWEPX4ESHQIEBPLIRQWFLKXMW6V3ZWZBS7LYMLDWZX34QIKA, seg_pos={0 0}, upload=55551]: Done uploading piece. err: ecclient: upload failed (node:12cFTycUosmf2ynRZmHYT1axN7VcS31wjqv74CxUpb5nThvmW5, address:104.36.82.53:4156): protocol: rpc: dial tcp 104.36.82.53:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:35:30.330734615 (58h0m13.175536147s) [noise=true , address=104.36.82.10:4091 , piece_id=RO37R6434DHQIIT6XOUXER7OIWLGRGDDSLCRPN4JHBHCVXQB2HUQ, seg_pos={0 0}, upload=55551]: Done uploading piece. err: ecclient: upload failed (node:1yyUzkigTS9m3bDrZtAFL7tciNxpiCNswBSfBqXn1UW83GMLwi, address:104.36.82.10:4091): protocol: write tcp 159.223.143.39:51804->104.36.82.10:4091: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:35:30.330761403 (58h0m13.17556293s) [noise=true , address=104.36.82.46:4506 , piece_id=5JHJHBQFHPBF26MTE3XOTCWZ63ZPORLVOZB7EBBXCRDR2N45AQ4A, seg_pos={0 0}, upload=55551]: Done uploading piece. err: ecclient: upload failed (node:12o9EEYka41UdTcVAmP7BJy8vrtQ2h4kvi1wGjcnedX4UKJspL9, address:104.36.82.46:4506): protocol: read tcp 159.223.143.39:42548->104.36.82.46:4506: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:35:30.330800203 (58h0m13.175601739s) [seg_pos={0 0}, upload=55551]: Done waiting for segment. successful: 8 optimal: 14 errs: <nil> Jul 26 21:35:30.356631027 (58h0m13.201432553s) [upload=55551]: Done uploading segments... Jul 26 21:35:30.356688827 (58h0m13.201490357s) [upload=55551]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 8) Duration: 1m0.033191468s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.072658574s FAIL (2025-07-26 21:35:22 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.072737722s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.523256756s FAIL (2025-07-26 21:34:38 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.523293841s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [......................................................XXXXXX] duration: 1m0.044248806s FAIL (2025-07-26 21:34:18 UTC) ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== .BatchRequestItem_ObjectBegin Jul 26 21:33:18.817599789 (57h58m1.662401312s) [upload=55528]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:33:18.870616476 (57h58m1.715418s) [seg_pos={0 0}, upload=55528]: Begin upload segment... Jul 26 21:33:18.870686311 (57h58m1.715487841s) [seg_pos={0 0}, upload=55528]: Done begin upload segment. Jul 26 21:33:18.870740384 (57h58m1.715541913s) [noise=true , address=79.127.201.215:4353 , piece_id=L3KMR7PHHNMULANXGZLVRJC76A3XFGBOZI4ZLVLENQ6DOF3BD5RA, seg_pos={0 0}, upload=55528]: Uploading piece... Jul 26 21:33:18.871158544 (57h58m1.715960076s) [noise=true , address=104.36.82.49:4981 , piece_id=VDO2MLHBJAP2NZ4AYKIFDT2FEK3MMYEVDPRB5J2TMHZNJB7NSLHA, seg_pos={0 0}, upload=55528]: Uploading piece... Jul 26 21:33:18.871483634 (57h58m1.716285161s) [noise=true , address=79.127.205.234:4614 , piece_id=6P5QXM5FL7EVIDOB6RN7HPDI5TK6UR7LWGBH3XSXDICW7FE7WLRA, seg_pos={0 0}, upload=55528]: Uploading piece... Jul 26 21:33:18.871762693 (57h58m1.716564218s) [noise=true , address=45.91.116.9:4269 , piece_id=XC4AD7EJXJZ6WYTPL2O32XC2MYWXE673RNL3AV6ITSCJNCICGNRA, seg_pos={0 0}, upload=55528]: Uploading piece... Jul 26 21:33:18.872023612 (57h58m1.716825137s) [noise=true , address=79.127.205.235:4679 , piece_id=B5DGLDBSIH64TUENQST7APMRJON6YJ6F4UODKIRG6VFISYGNLWIA, seg_pos={0 0}, upload=55528]: Uploading piece... Jul 26 21:33:18.872461004 (57h58m1.717262532s) [noise=true , address=104.36.82.56:4256 , piece_id=RV57SDZQ3VSZEWWHRZSXEJGPMLRDSHYSISTQJR3QBP5RCCRQL6WQ, seg_pos={0 0}, upload=55528]: Uploading piece... Jul 26 21:33:18.872761099 (57h58m1.717562621s) [noise=true , address=104.36.82.15:4284 , piece_id=ENKFM6A2V2LTHFQ5YXXSL2KT7V36RPYH7GMDVPD6ALT26UEASAYQ, seg_pos={0 0}, upload=55528]: Uploading piece... Jul 26 21:33:18.873026744 (57h58m1.717828267s) [noise=true , address=104.36.82.51:4064 , piece_id=OKTBKSIXQA6R56I3TZB7ZLBKFFCPQZ2APOO6DDBTLCVBMJ55TNVA, seg_pos={0 0}, upload=55528]: Uploading piece... Jul 26 21:33:18.873314049 (57h58m1.718115577s) [noise=true , address=104.36.82.53:4133 , piece_id=ZOSPQBMHQKISMXEP7FYXN47NYQT5UDFVHHDA7DI5KPL3GZ7DHWPA, seg_pos={0 0}, upload=55528]: Uploading piece... Jul 26 21:33:18.879074178 (57h58m1.723875711s) [noise=true , address=104.36.82.16:4300 , piece_id=YQI3MIUUODNZIHPUQMK2A2WZI3K4FANGW7LQBXM3AHA6X65AUSKQ, seg_pos={0 0}, upload=55528]: Uploading piece... Jul 26 21:33:18.879526493 (57h58m1.724328021s) [noise=true , address=79.127.219.38:4006 , piece_id=4ZJA3EHCU2A4SGOL6HUMZHCBORRYXSTI3Q7NI7A7PPGGBNBVG5JQ, seg_pos={0 0}, upload=55528]: Uploading piece... Jul 26 21:33:18.879840049 (57h58m1.724641574s) [noise=true , address=79.127.205.246:4278 , piece_id=63IXKF6RZLMOZM5GM36SGHOB3DRSJLD2U32WNYUXEKSJQFW2FILQ, seg_pos={0 0}, upload=55528]: Uploading piece... Jul 26 21:33:18.880133308 (57h58m1.724934835s) [noise=true , address=104.36.82.46:4494 , piece_id=W3WZEO5RZVED4ETXNMLETYJNTFIEBYSU72653KMC63TOGHCFTHEA, seg_pos={0 0}, upload=55528]: Uploading piece... Jul 26 21:33:18.880467172 (57h58m1.725268704s) [noise=true , address=64.49.13.140:4509 , piece_id=PH45WUI33HNFE2CL5536IQYTF3D3XO7N3X5P26UZRIOT37HHUVAQ, seg_pos={0 0}, upload=55528]: Uploading piece... Jul 26 21:33:18.880770872 (57h58m1.725572398s) [noise=true , address=104.36.82.13:4206 , piece_id=X5ORVLL7RN3EUK2BOAYG5WYFT5ZE2XCIIO33KMW4EOAR2TBKPXJA, seg_pos={0 0}, upload=55528]: Uploading piece... Jul 26 21:33:18.881036601 (57h58m1.725838126s) [noise=true , address=104.36.82.47:4538 , piece_id=6V67JFOP6ZHEQ4773SX6UQ6ZTYWQBI3ASM557TQDMZKHWQ2ZZSWA, seg_pos={0 0}, upload=55528]: Uploading piece... Jul 26 21:33:18.881328760 (57h58m1.726130285s) [noise=true , address=77.90.31.21:4910 , piece_id=KU5GFQ2IVUBF2BD2UJZWV2LYLB64HXTV3THH7KSLJCY5LFWIK4DQ, seg_pos={0 0}, upload=55528]: Uploading piece... Jul 26 21:33:18.881613548 (57h58m1.726415074s) [noise=true , address=156.146.43.225:4448 , piece_id=BDCHMOK5Y6ISZYBLZZ43OBPWUGZHLJZ2WQ3H3AG5R77H5HX5EF5A, seg_pos={0 0}, upload=55528]: Uploading piece... Jul 26 21:33:18.950544275 (57h58m1.795345799s) [upload=55528]: Next returned nil segment Jul 26 21:33:18.950562049 (57h58m1.795363562s) [upload=55528]: Waiting for error group managing segments... Jul 26 21:33:19.178128159 (57h58m2.022929674s) [noise=true , address=79.127.219.38:4006 , piece_id=4ZJA3EHCU2A4SGOL6HUMZHCBORRYXSTI3Q7NI7A7PPGGBNBVG5JQ, seg_pos={0 0}, upload=55528]: Done uploading piece. err: <nil> Jul 26 21:33:19.599375516 (57h58m2.444177032s) [noise=true , address=79.127.205.246:4278 , piece_id=63IXKF6RZLMOZM5GM36SGHOB3DRSJLD2U32WNYUXEKSJQFW2FILQ, seg_pos={0 0}, upload=55528]: Done uploading piece. err: <nil> Jul 26 21:33:19.983663139 (57h58m2.828464661s) [noise=true , address=156.146.43.225:4448 , piece_id=BDCHMOK5Y6ISZYBLZZ43OBPWUGZHLJZ2WQ3H3AG5R77H5HX5EF5A, seg_pos={0 0}, upload=55528]: Done uploading piece. err: <nil> Jul 26 21:33:20.000794289 (57h58m2.84559585s) [noise=true , address=79.127.205.234:4614 , piece_id=6P5QXM5FL7EVIDOB6RN7HPDI5TK6UR7LWGBH3XSXDICW7FE7WLRA, seg_pos={0 0}, upload=55528]: Done uploading piece. err: <nil> Jul 26 21:33:20.007923895 (57h58m2.85272544s) [noise=true , address=79.127.205.235:4679 , piece_id=B5DGLDBSIH64TUENQST7APMRJON6YJ6F4UODKIRG6VFISYGNLWIA, seg_pos={0 0}, upload=55528]: Done uploading piece. err: <nil> Jul 26 21:33:20.185559612 (57h58m3.030361136s) [noise=true , address=77.90.31.21:4910 , piece_id=KU5GFQ2IVUBF2BD2UJZWV2LYLB64HXTV3THH7KSLJCY5LFWIK4DQ, seg_pos={0 0}, upload=55528]: Done uploading piece. err: <nil> Jul 26 21:33:20.327685565 (57h58m3.172487089s) [noise=true , address=79.127.201.215:4353 , piece_id=L3KMR7PHHNMULANXGZLVRJC76A3XFGBOZI4ZLVLENQ6DOF3BD5RA, seg_pos={0 0}, upload=55528]: Done uploading piece. err: <nil> Jul 26 21:33:22.165102472 (57h58m5.009903999s) [noise=true , address=45.91.116.9:4269 , piece_id=XC4AD7EJXJZ6WYTPL2O32XC2MYWXE673RNL3AV6ITSCJNCICGNRA, seg_pos={0 0}, upload=55528]: Done uploading piece. err: <nil> Jul 26 21:33:22.827642336 (57h58m5.672443852s) [noise=true , address=64.49.13.140:4509 , piece_id=PH45WUI33HNFE2CL5536IQYTF3D3XO7N3X5P26UZRIOT37HHUVAQ, seg_pos={0 0}, upload=55528]: Done uploading piece. err: <nil> Jul 26 21:33:31.291559088 (57h58m14.136360621s) [noise=true , address=104.36.82.13:4206 , piece_id=X5ORVLL7RN3EUK2BOAYG5WYFT5ZE2XCIIO33KMW4EOAR2TBKPXJA, seg_pos={0 0}, upload=55528]: Done uploading piece. err: <nil> Jul 26 21:33:44.272857734 (57h58m27.117659257s) [noise=true , address=104.36.82.16:4300 , piece_id=YQI3MIUUODNZIHPUQMK2A2WZI3K4FANGW7LQBXM3AHA6X65AUSKQ, seg_pos={0 0}, upload=55528]: Done uploading piece. err: <nil> Jul 26 21:34:14.360841027 (57h58m57.205642537s) [noise=true , address=104.36.82.15:4284 , piece_id=ENKFM6A2V2LTHFQ5YXXSL2KT7V36RPYH7GMDVPD6ALT26UEASAYQ, seg_pos={0 0}, upload=55528]: Done uploading piece. err: <nil> Jul 26 21:34:18.785962306 (57h59m1.630763819s) [noise=true , address=104.36.82.56:4256 , piece_id=RV57SDZQ3VSZEWWHRZSXEJGPMLRDSHYSISTQJR3QBP5RCCRQL6WQ, seg_pos={0 0}, upload=55528]: Done uploading piece. err: ecclient: upload failed (node:11N1BR6rgiYye7CnuM5xDJogUXoUp4FgXrXM41GdhfrRCbbgT8, address:104.36.82.56:4256): protocol: context deadline exceeded; context deadline exceeded Jul 26 21:34:18.786077386 (57h59m1.630878901s) [noise=true , address=104.36.82.47:4538 , piece_id=6V67JFOP6ZHEQ4773SX6UQ6ZTYWQBI3ASM557TQDMZKHWQ2ZZSWA, seg_pos={0 0}, upload=55528]: Done uploading piece. err: ecclient: upload failed (node:12P3N79R3vpvkpsJ4fEd6NcJtmMpszRW5KHgfYJvkHQzvSTf3i5, address:104.36.82.47:4538): protocol: write tcp 159.223.143.39:47262->104.36.82.47:4538: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:34:18.786236365 (57h59m1.631037881s) [noise=true , address=104.36.82.51:4064 , piece_id=OKTBKSIXQA6R56I3TZB7ZLBKFFCPQZ2APOO6DDBTLCVBMJ55TNVA, seg_pos={0 0}, upload=55528]: Done uploading piece. err: ecclient: upload failed (node:1HtP28UAricpA5ZBZrn2eLK1GBQxFGL2UNchDUFViqMhw4pXjy, address:104.36.82.51:4064): protocol: read tcp 159.223.143.39:52182->104.36.82.51:4064: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:34:18.786269862 (57h59m1.631071377s) [noise=true , address=104.36.82.53:4133 , piece_id=ZOSPQBMHQKISMXEP7FYXN47NYQT5UDFVHHDA7DI5KPL3GZ7DHWPA, seg_pos={0 0}, upload=55528]: Done uploading piece. err: ecclient: upload failed (node:12usLZNRYviwS7dfk6soCgE7XP5LeyVcSUpjhWAk3stm61tjR7g, address:104.36.82.53:4133): protocol: write tcp 159.223.143.39:40444->104.36.82.53:4133: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:34:18.786304277 (57h59m1.631105805s) [noise=true , address=104.36.82.49:4981 , piece_id=VDO2MLHBJAP2NZ4AYKIFDT2FEK3MMYEVDPRB5J2TMHZNJB7NSLHA, seg_pos={0 0}, upload=55528]: Done uploading piece. err: ecclient: upload failed (node:1u49RC6mHSy43oo9REosyNVdFB3jKk98A6hH4j9qgCQaF5qPSv, address:104.36.82.49:4981): protocol: write tcp 159.223.143.39:43838->104.36.82.49:4981: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:34:18.786354247 (57h59m1.63115576s) [noise=true , address=104.36.82.46:4494 , piece_id=W3WZEO5RZVED4ETXNMLETYJNTFIEBYSU72653KMC63TOGHCFTHEA, seg_pos={0 0}, upload=55528]: Done uploading piece. err: ecclient: upload failed (node:17b8LdpavJCCFXEBj7ndUKR6sge6Ntv96cGK5TmahF3MtwTV9z, address:104.36.82.46:4494): protocol: write tcp 159.223.143.39:51888->104.36.82.46:4494: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:34:18.786380823 (57h59m1.631182346s) [seg_pos={0 0}, upload=55528]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Jul 26 21:34:18.828529173 (57h59m1.67333069s) [upload=55528]: Done uploading segments... Jul 26 21:34:18.828554458 (57h59m1.673355981s) [upload=55528]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 1m0.044266587s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.400434281s FAIL (2025-07-26 21:34:11 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.400465676s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [.......................................................XXXXX] duration: 1m0.03410632s FAIL (2025-07-26 21:33:18 UTC) ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 7) ======LOGS====== ) [noise=true , address=104.36.82.9:4014 , piece_id=RRVF4BPFJHF7AHSZNSZZ4FYAVTHK3X4VN637XDAW7NDYEYXBY2OA, seg_pos={0 0}, upload=55515]: Uploading piece... Jul 26 21:32:18.771169556 (57h57m1.615971089s) [noise=true , address=79.127.220.101:4048 , piece_id=7B5JJRR63VTEJOHO7ZGX6FPIJJEDV7Z6CHOTHLSH5L54GV54DUZA, seg_pos={0 0}, upload=55515]: Uploading piece... Jul 26 21:32:18.771474869 (57h57m1.616276401s) [noise=true , address=104.36.82.6:4903 , piece_id=M27M4CYGH6DG4BQD34JVEFDYZSKCLZ45DCULIJJM2B7N27D5NUZQ, seg_pos={0 0}, upload=55515]: Uploading piece... Jul 26 21:32:18.771630870 (57h57m1.616432398s) [noise=true , address=104.36.82.54:4173 , piece_id=L267AE6HTT2JRJR3ELQ6WNMCM3WVWVKSJSIDOVJXGYL2RAXP55YQ, seg_pos={0 0}, upload=55515]: Uploading piece... Jul 26 21:32:18.771864806 (57h57m1.616666338s) [noise=true , address=104.36.82.46:4505 , piece_id=XEOVKSSH57PV2XRLYUVGBSGHXBYUQOT7GAYINM4ES7PPAYDKLEMQ, seg_pos={0 0}, upload=55515]: Uploading piece... Jul 26 21:32:18.772014945 (57h57m1.61681647s) [noise=true , address=79.127.219.47:4067 , piece_id=H3PU34WDBXB5KFBMO3IAUOSZ6BSNMW65NWF5MA3AATDFGK22EUZA, seg_pos={0 0}, upload=55515]: Uploading piece... Jul 26 21:32:18.772367949 (57h57m1.617169482s) [noise=true , address=104.36.82.56:4255 , piece_id=UOIZZJODFW7L2IE4OIYVFCVDT6W5JXGYGTLO7BYDB23L75BMEIGQ, seg_pos={0 0}, upload=55515]: Uploading piece... Jul 26 21:32:18.772618146 (57h57m1.617419679s) [noise=true , address=104.36.82.44:4870 , piece_id=JIRIP3TA32GUYIFOCJB2UE3QSHUKVOSUFVCKJCXNX3BO7LQZVI7Q, seg_pos={0 0}, upload=55515]: Uploading piece... Jul 26 21:32:18.772726956 (57h57m1.617528466s) [noise=true , address=104.36.82.19:4408 , piece_id=PHGLTNU2XIPLGLNEF47BUCZPYG2RNHD5Y2ZMFIVX5Y3JUN57P4NQ, seg_pos={0 0}, upload=55515]: Uploading piece... Jul 26 21:32:18.773108160 (57h57m1.61790969s) [noise=true , address=104.36.82.7:4914 , piece_id=FNWZ6DNITKBWNIEFA2EL6327ELTD6UE7UWIEXRW56VG5MIQNE2SQ, seg_pos={0 0}, upload=55515]: Uploading piece... Jul 26 21:32:18.773588153 (57h57m1.618389675s) [noise=true , address=77.90.31.17:4792 , piece_id=D6USQURSXGH5MHZXDZNHY2U5PPEOBELBIG2GTPDXBHNGKR4OK3OA, seg_pos={0 0}, upload=55515]: Uploading piece... Jul 26 21:32:18.774039303 (57h57m1.618840831s) [noise=true , address=104.36.82.52:4108 , piece_id=HISHXL7GNAHNTXGAMUDMMUACJ6NKEVB3H32JOJVX62UYGSIO2AJA, seg_pos={0 0}, upload=55515]: Uploading piece... Jul 26 21:32:18.774425810 (57h57m1.61922734s) [noise=true , address=104.36.82.8:4989 , piece_id=GNFJO2VJLVOQX52RUVSSHR7UDZIGZF5VLMQ4YDDI2ZPOQVXZ4YMQ, seg_pos={0 0}, upload=55515]: Uploading piece... Jul 26 21:32:18.774766382 (57h57m1.619567918s) [noise=true , address=104.36.82.47:4542 , piece_id=WQCJNA35T6HXJIQDMTOY2QHUTMHNPJEBVU5HNIZL2UJRH3WWS43Q, seg_pos={0 0}, upload=55515]: Uploading piece... Jul 26 21:32:18.775167990 (57h57m1.61996952s) [noise=true , address=104.36.82.12:4175 , piece_id=WPPCZAN7RT2NINYIMOE2TQCEMGCKFHVWOW2WE4546H3W5TRETV3Q, seg_pos={0 0}, upload=55515]: Uploading piece... Jul 26 21:32:18.843481685 (57h57m1.688283219s) [upload=55515]: Next returned nil segment Jul 26 21:32:18.843502008 (57h57m1.688303571s) [upload=55515]: Waiting for error group managing segments... Jul 26 21:32:18.849082462 (57h57m1.693883972s) [noise=true , address=79.127.219.47:4067 , piece_id=H3PU34WDBXB5KFBMO3IAUOSZ6BSNMW65NWF5MA3AATDFGK22EUZA, seg_pos={0 0}, upload=55515]: Done uploading piece. err: <nil> Jul 26 21:32:18.942921682 (57h57m1.787723202s) [noise=true , address=79.127.220.101:4048 , piece_id=7B5JJRR63VTEJOHO7ZGX6FPIJJEDV7Z6CHOTHLSH5L54GV54DUZA, seg_pos={0 0}, upload=55515]: Done uploading piece. err: <nil> Jul 26 21:32:18.976773483 (57h57m1.821574997s) [noise=true , address=104.36.82.5:4851 , piece_id=TRQ7ZNZXM26FXIT3FWBBNKATKWMXMB6REZSDA7VPWWHR7QANZNEA, seg_pos={0 0}, upload=55515]: Done uploading piece. err: <nil> Jul 26 21:32:19.054236448 (57h57m1.899037965s) [noise=true , address=79.127.219.51:4211 , piece_id=6GWVZQ3E5W7VCNEJIBBFV3B6WDFW5V7HN5HQDILY2G2PID35IFVA, seg_pos={0 0}, upload=55515]: Done uploading piece. err: <nil> Jul 26 21:32:19.753585952 (57h57m2.598387476s) [noise=true , address=79.127.201.215:4353 , piece_id=RKFO5VXI4U3GC7YWUIHWTFNKZ52L3HXVR76UOTTSJ4MMKM2SXCIQ, seg_pos={0 0}, upload=55515]: Done uploading piece. err: <nil> Jul 26 21:32:20.078566679 (57h57m2.923368205s) [noise=true , address=77.90.31.17:4792 , piece_id=D6USQURSXGH5MHZXDZNHY2U5PPEOBELBIG2GTPDXBHNGKR4OK3OA, seg_pos={0 0}, upload=55515]: Done uploading piece. err: <nil> Jul 26 21:32:24.137362653 (57h57m6.982164178s) [noise=true , address=104.36.82.12:4175 , piece_id=WPPCZAN7RT2NINYIMOE2TQCEMGCKFHVWOW2WE4546H3W5TRETV3Q, seg_pos={0 0}, upload=55515]: Done uploading piece. err: <nil> Jul 26 21:33:18.695228426 (57h58m1.540029948s) [noise=true , address=104.36.82.46:4505 , piece_id=XEOVKSSH57PV2XRLYUVGBSGHXBYUQOT7GAYINM4ES7PPAYDKLEMQ, seg_pos={0 0}, upload=55515]: Done uploading piece. err: ecclient: upload failed (node:1i6itGoBhLsLiqC8et8Y5L62fLjcWxy18u39NAr4R3nXNMWcsf, address:104.36.82.46:4505): protocol: rpc: dial tcp 104.36.82.46:4505: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:18.695272310 (57h58m1.540073848s) [noise=true , address=104.36.82.9:4014 , piece_id=RRVF4BPFJHF7AHSZNSZZ4FYAVTHK3X4VN637XDAW7NDYEYXBY2OA, seg_pos={0 0}, upload=55515]: Done uploading piece. err: ecclient: upload failed (node:12uK9ULUdABDm5XBP4nh1omWhR5qoQerFsCmL4bQ9dmNespXpTy, address:104.36.82.9:4014): protocol: rpc: dial tcp 104.36.82.9:4014: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:18.695299418 (57h58m1.54010094s) [noise=true , address=104.36.82.44:4870 , piece_id=JIRIP3TA32GUYIFOCJB2UE3QSHUKVOSUFVCKJCXNX3BO7LQZVI7Q, seg_pos={0 0}, upload=55515]: Done uploading piece. err: ecclient: upload failed (node:12sDH7ccMQhjePqpbTkkxMAtumNLxpMHiT56cnfaYrUTFYxuPVt, address:104.36.82.44:4870): protocol: rpc: dial tcp 104.36.82.44:4870: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:18.695338521 (57h58m1.540140044s) [noise=true , address=104.36.82.6:4903 , piece_id=M27M4CYGH6DG4BQD34JVEFDYZSKCLZ45DCULIJJM2B7N27D5NUZQ, seg_pos={0 0}, upload=55515]: Done uploading piece. err: ecclient: upload failed (node:1zFBuDaxAtbFho2wumoXWSoQbB7y9VaSFizz31gV8n1fLK6syE, address:104.36.82.6:4903): protocol: rpc: dial tcp 104.36.82.6:4903: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:18.695352071 (57h58m1.540153594s) [noise=true , address=104.36.82.19:4408 , piece_id=PHGLTNU2XIPLGLNEF47BUCZPYG2RNHD5Y2ZMFIVX5Y3JUN57P4NQ, seg_pos={0 0}, upload=55515]: Done uploading piece. err: ecclient: upload failed (node:12evzqLGVaxayKpKejqBXNLwqTKyKNKa94NZYJBUCdvyUo3vhGJ, address:104.36.82.19:4408): protocol: rpc: dial tcp 104.36.82.19:4408: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:18.695395096 (57h58m1.540196608s) [noise=true , address=104.36.82.52:4108 , piece_id=HISHXL7GNAHNTXGAMUDMMUACJ6NKEVB3H32JOJVX62UYGSIO2AJA, seg_pos={0 0}, upload=55515]: Done uploading piece. err: ecclient: upload failed (node:12XJJZCUSHUS52YC6jhoPB4ayjsyKZT477PymPj4inkZtdUvheT, address:104.36.82.52:4108): protocol: rpc: dial tcp 104.36.82.52:4108: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:18.695407644 (57h58m1.540209168s) [noise=true , address=104.36.82.8:4989 , piece_id=GNFJO2VJLVOQX52RUVSSHR7UDZIGZF5VLMQ4YDDI2ZPOQVXZ4YMQ, seg_pos={0 0}, upload=55515]: Done uploading piece. err: ecclient: upload failed (node:14QoLDSHxoQN2FBG8w5kQTxUvtpvnGNZf5qgJGpGB43eTBJ511, address:104.36.82.8:4989): protocol: rpc: dial tcp 104.36.82.8:4989: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:18.695458184 (57h58m1.540259698s) [noise=true , address=104.36.82.47:4542 , piece_id=WQCJNA35T6HXJIQDMTOY2QHUTMHNPJEBVU5HNIZL2UJRH3WWS43Q, seg_pos={0 0}, upload=55515]: Done uploading piece. err: ecclient: upload failed (node:129FwaCA1FbzJbmRNzhQeVgygicJYHWkSS9fLnJbAG7KZPpokLU, address:104.36.82.47:4542): protocol: rpc: dial tcp 104.36.82.47:4542: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:18.695463578 (57h58m1.540265107s) [noise=true , address=104.36.82.56:4255 , piece_id=UOIZZJODFW7L2IE4OIYVFCVDT6W5JXGYGTLO7BYDB23L75BMEIGQ, seg_pos={0 0}, upload=55515]: Done uploading piece. err: ecclient: upload failed (node:12S6rwimXBvaEb4KneAwL5H3t5zTmvGVhyJSWCTU1u17r4RWJta, address:104.36.82.56:4255): protocol: rpc: dial tcp 104.36.82.56:4255: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:18.695493807 (57h58m1.540295329s) [noise=true , address=104.36.82.7:4914 , piece_id=FNWZ6DNITKBWNIEFA2EL6327ELTD6UE7UWIEXRW56VG5MIQNE2SQ, seg_pos={0 0}, upload=55515]: Done uploading piece. err: ecclient: upload failed (node:1MArPyMrdvyw4VFuArmattTtneNTaN5mt82nfGwY2sKHvQnL5D, address:104.36.82.7:4914): protocol: rpc: dial tcp 104.36.82.7:4914: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:18.695549964 (57h58m1.540351499s) [noise=true , address=104.36.82.54:4173 , piece_id=L267AE6HTT2JRJR3ELQ6WNMCM3WVWVKSJSIDOVJXGYL2RAXP55YQ, seg_pos={0 0}, upload=55515]: Done uploading piece. err: ecclient: upload failed (node:1w2v4iyYtJtcdz7shYhG8CjCDQ1dWjeJVwY5Ubi7Ex5Y38e2i1, address:104.36.82.54:4173): protocol: write tcp 159.223.143.39:44386->104.36.82.54:4173: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:18.695585224 (57h58m1.540386748s) [seg_pos={0 0}, upload=55515]: Done waiting for segment. successful: 7 optimal: 14 errs: <nil> Jul 26 21:33:18.726671453 (57h58m1.571472987s) [upload=55515]: Done uploading segments... Jul 26 21:33:18.726720539 (57h58m1.571522069s) [upload=55515]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 7) Duration: 1m0.034131572s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 665.084873ms FAIL (2025-07-26 21:33:04 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 665.119199ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [........................................................XXXX] duration: 1m0.026623443s FAIL (2025-07-26 21:32:18 UTC) ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== , upload=55498]: Begin upload segment... Jul 26 21:31:18.670473950 (57h56m1.515275479s) [seg_pos={0 0}, upload=55498]: Done begin upload segment. Jul 26 21:31:18.670521660 (57h56m1.515323184s) [noise=true , address=104.36.82.37:4666 , piece_id=B5V52A3DIRW4ZQGHMX47GXCOBWDCG74PNCK6DZDY3IELDE66B5LQ, seg_pos={0 0}, upload=55498]: Uploading piece... Jul 26 21:31:18.671076797 (57h56m1.515878345s) [noise=true , address=104.36.82.52:4085 , piece_id=X6PR5WJM4MYUSSIT5UD4UQIYSWUIUCIRGWI2PDDR3YUTZBVP4PLA, seg_pos={0 0}, upload=55498]: Uploading piece... Jul 26 21:31:18.671095791 (57h56m1.515897323s) [noise=true , address=79.127.201.215:4353 , piece_id=YM2JX3VODF3GJRAFO63NRHF2EZQMQ626NNB76WI7DNWFGSADYKUQ, seg_pos={0 0}, upload=55498]: Uploading piece... Jul 26 21:31:18.671190890 (57h56m1.515992423s) [noise=true , address=104.36.82.44:4878 , piece_id=IKBMGP2VOM4BF2T4TP5ZLQS423CLBN5BBZEQXLVJBEOYSPK3A65A, seg_pos={0 0}, upload=55498]: Uploading piece... Jul 26 21:31:18.671590899 (57h56m1.516392427s) [noise=true , address=104.36.82.55:4215 , piece_id=KN7IQQUNFXJCT4WSDRX2AQTT4ZEUUKGKTN3Q7UFU24KAESK2FSSQ, seg_pos={0 0}, upload=55498]: Uploading piece... Jul 26 21:31:18.671603910 (57h56m1.516405443s) [noise=true , address=104.36.82.13:4189 , piece_id=PKDOMBWGOJBI2KIV2KPDP4YUIFL674DIUSKCZZDBEV5O7AIQLVLA, seg_pos={0 0}, upload=55498]: Uploading piece... Jul 26 21:31:18.671950913 (57h56m1.516752438s) [noise=true , address=156.146.43.225:4449 , piece_id=XU3VDIUD43JK5GJ7FQGO46MHNZTCHCOSVIF7YEBPRGGST5FTHDKA, seg_pos={0 0}, upload=55498]: Uploading piece... Jul 26 21:31:18.672076505 (57h56m1.516878035s) [noise=true , address=79.127.205.235:4676 , piece_id=KTS3FW6U3IT5FRYZPGXOK2LYRGBEAPPJDONYOT23EKFTLN3UHHGQ, seg_pos={0 0}, upload=55498]: Uploading piece... Jul 26 21:31:18.672467605 (57h56m1.517269136s) [noise=true , address=64.49.13.138:4482 , piece_id=4SHXAVOJ7MPDL5DTE2CJ3QCFIIDQL5WFZSJS4GGB73XSD2MVBOJQ, seg_pos={0 0}, upload=55498]: Uploading piece... Jul 26 21:31:18.672638931 (57h56m1.517440456s) [noise=true , address=79.127.205.234:4608 , piece_id=IQHQBMZZKDCX2KTQAKK5KPQY67IQLXLFNSD6X5ISAVNSR6TRSPBQ, seg_pos={0 0}, upload=55498]: Uploading piece... Jul 26 21:31:18.672826743 (57h56m1.517628274s) [noise=true , address=45.91.116.14:4206 , piece_id=WDM5BJTZW6MARTL7HDOGUARK4ZIPFI3MB5ZRPYSOIBTNH44NSJ4Q, seg_pos={0 0}, upload=55498]: Uploading piece... Jul 26 21:31:18.673055312 (57h56m1.517856844s) [noise=true , address=104.36.82.54:4167 , piece_id=E2QB3QMNMNRAMRX6JWYSHSM5ZSMIUEUOIY2UTWGMAH7L2TRTEA7Q, seg_pos={0 0}, upload=55498]: Uploading piece... Jul 26 21:31:18.673198266 (57h56m1.517999798s) [noise=true , address=104.36.82.53:4153 , piece_id=SVBJCGYJWJDP7GXLD4AN5M3D2JJALVHZNJVUHL4ZNZ3WCFPPJY2Q, seg_pos={0 0}, upload=55498]: Uploading piece... Jul 26 21:31:18.673518060 (57h56m1.51831961s) [noise=true , address=104.36.82.38:4690 , piece_id=D62TB7FO3FBFMPT4H6REJQ7O326TOB7ZXW22UEGR3LJ7RY6HLQJQ, seg_pos={0 0}, upload=55498]: Uploading piece... Jul 26 21:31:18.673556212 (57h56m1.518357746s) [noise=true , address=104.36.82.18:4379 , piece_id=7QMGGA327EIAOLR5BZKFFD2KNJMDEVJORM36GMBGFUHSYDAYQIDA, seg_pos={0 0}, upload=55498]: Uploading piece... Jul 26 21:31:18.673913273 (57h56m1.51871485s) [noise=true , address=104.36.82.10:4092 , piece_id=RS4Z6BCUSS7VRT7G7OWPWJWG7IV3U6ERCRT35LLBO2UHN54PS53A, seg_pos={0 0}, upload=55498]: Uploading piece... Jul 26 21:31:18.673945610 (57h56m1.518747144s) [noise=true , address=77.90.31.16:4747 , piece_id=GM5FVTM2S4LLGILE5RP427LL43M45YIES47M4VBKF76O37NBNBPA, seg_pos={0 0}, upload=55498]: Uploading piece... Jul 26 21:31:18.674357256 (57h56m1.519158791s) [noise=true , address=104.36.82.51:4074 , piece_id=37UJNJF2MVUITPD43IDHB5FHZPTYZTUR5LWF756JFIYM4GGSG5UA, seg_pos={0 0}, upload=55498]: Uploading piece... Jul 26 21:31:18.748348110 (57h56m1.593149625s) [upload=55498]: Next returned nil segment Jul 26 21:31:18.748423437 (57h56m1.593225017s) [upload=55498]: Waiting for error group managing segments... Jul 26 21:31:18.796358401 (57h56m1.641159917s) [noise=true , address=45.91.116.14:4206 , piece_id=WDM5BJTZW6MARTL7HDOGUARK4ZIPFI3MB5ZRPYSOIBTNH44NSJ4Q, seg_pos={0 0}, upload=55498]: Done uploading piece. err: <nil> Jul 26 21:31:19.104667790 (57h56m1.949469304s) [noise=true , address=156.146.43.225:4449 , piece_id=XU3VDIUD43JK5GJ7FQGO46MHNZTCHCOSVIF7YEBPRGGST5FTHDKA, seg_pos={0 0}, upload=55498]: Done uploading piece. err: <nil> Jul 26 21:31:19.203829487 (57h56m2.048631011s) [noise=true , address=79.127.201.215:4353 , piece_id=YM2JX3VODF3GJRAFO63NRHF2EZQMQ626NNB76WI7DNWFGSADYKUQ, seg_pos={0 0}, upload=55498]: Done uploading piece. err: <nil> Jul 26 21:31:19.283011596 (57h56m2.127813108s) [noise=true , address=79.127.205.235:4676 , piece_id=KTS3FW6U3IT5FRYZPGXOK2LYRGBEAPPJDONYOT23EKFTLN3UHHGQ, seg_pos={0 0}, upload=55498]: Done uploading piece. err: <nil> Jul 26 21:31:19.304825514 (57h56m2.14962703s) [noise=true , address=79.127.205.234:4608 , piece_id=IQHQBMZZKDCX2KTQAKK5KPQY67IQLXLFNSD6X5ISAVNSR6TRSPBQ, seg_pos={0 0}, upload=55498]: Done uploading piece. err: <nil> Jul 26 21:31:19.577547971 (57h56m2.422349503s) [noise=true , address=104.36.82.18:4379 , piece_id=7QMGGA327EIAOLR5BZKFFD2KNJMDEVJORM36GMBGFUHSYDAYQIDA, seg_pos={0 0}, upload=55498]: Done uploading piece. err: <nil> Jul 26 21:31:20.186859671 (57h56m3.031661199s) [noise=true , address=77.90.31.16:4747 , piece_id=GM5FVTM2S4LLGILE5RP427LL43M45YIES47M4VBKF76O37NBNBPA, seg_pos={0 0}, upload=55498]: Done uploading piece. err: <nil> Jul 26 21:31:28.327429697 (57h56m11.172231232s) [noise=true , address=64.49.13.138:4482 , piece_id=4SHXAVOJ7MPDL5DTE2CJ3QCFIIDQL5WFZSJS4GGB73XSD2MVBOJQ, seg_pos={0 0}, upload=55498]: Done uploading piece. err: <nil> Jul 26 21:31:34.617899700 (57h56m17.462701217s) [noise=true , address=104.36.82.51:4074 , piece_id=37UJNJF2MVUITPD43IDHB5FHZPTYZTUR5LWF756JFIYM4GGSG5UA, seg_pos={0 0}, upload=55498]: Done uploading piece. err: <nil> Jul 26 21:31:59.504719247 (57h56m42.349520772s) [noise=true , address=104.36.82.37:4666 , piece_id=B5V52A3DIRW4ZQGHMX47GXCOBWDCG74PNCK6DZDY3IELDE66B5LQ, seg_pos={0 0}, upload=55498]: Done uploading piece. err: <nil> Jul 26 21:32:05.757189299 (57h56m48.601990821s) [noise=true , address=104.36.82.38:4690 , piece_id=D62TB7FO3FBFMPT4H6REJQ7O326TOB7ZXW22UEGR3LJ7RY6HLQJQ, seg_pos={0 0}, upload=55498]: Done uploading piece. err: <nil> Jul 26 21:32:18.596525051 (57h57m1.441326577s) [noise=true , address=104.36.82.44:4878 , piece_id=IKBMGP2VOM4BF2T4TP5ZLQS423CLBN5BBZEQXLVJBEOYSPK3A65A, seg_pos={0 0}, upload=55498]: Done uploading piece. err: ecclient: upload failed (node:1wX9xRmyv3KWVyJhFmgBt9uYqDU6U768dqHagKZeKBSEnFup5y, address:104.36.82.44:4878): protocol: rpc: dial tcp 104.36.82.44:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:32:18.596572212 (57h57m1.441373742s) [noise=true , address=104.36.82.52:4085 , piece_id=X6PR5WJM4MYUSSIT5UD4UQIYSWUIUCIRGWI2PDDR3YUTZBVP4PLA, seg_pos={0 0}, upload=55498]: Done uploading piece. err: ecclient: upload failed (node:17x7QtnZsENMZPU3tqkh8jEd29JdowLvT855W4kuourgUA6y5C, address:104.36.82.52:4085): protocol: rpc: dial tcp 104.36.82.52:4085: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:32:18.596664550 (57h57m1.441466075s) [noise=true , address=104.36.82.53:4153 , piece_id=SVBJCGYJWJDP7GXLD4AN5M3D2JJALVHZNJVUHL4ZNZ3WCFPPJY2Q, seg_pos={0 0}, upload=55498]: Done uploading piece. err: ecclient: upload failed (node:1fPp4DvLe18dZWPmMPFiC8o9jL4uiEFNmPAHHmFEH74ZXNndw3, address:104.36.82.53:4153): protocol: rpc: dial tcp 104.36.82.53:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:32:18.596743217 (57h57m1.441544741s) [noise=true , address=104.36.82.54:4167 , piece_id=E2QB3QMNMNRAMRX6JWYSHSM5ZSMIUEUOIY2UTWGMAH7L2TRTEA7Q, seg_pos={0 0}, upload=55498]: Done uploading piece. err: ecclient: upload failed (node:1hhT46NW5CQ7dWHRxy5MSjyKc3PiQqBifThT7dBmx2sgNvn8GZ, address:104.36.82.54:4167): protocol: write tcp 159.223.143.39:37996->104.36.82.54:4167: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:32:18.596801870 (57h57m1.441603398s) [noise=true , address=104.36.82.13:4189 , piece_id=PKDOMBWGOJBI2KIV2KPDP4YUIFL674DIUSKCZZDBEV5O7AIQLVLA, seg_pos={0 0}, upload=55498]: Done uploading piece. err: ecclient: upload failed (node:1jH3757i2F3q18XSpoXznsysZPaXmEnqynpLPDupgRfxJrTgKv, address:104.36.82.13:4189): protocol: write tcp 159.223.143.39:53450->104.36.82.13:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:32:18.596843007 (57h57m1.441644538s) [noise=true , address=104.36.82.55:4215 , piece_id=KN7IQQUNFXJCT4WSDRX2AQTT4ZEUUKGKTN3Q7UFU24KAESK2FSSQ, seg_pos={0 0}, upload=55498]: Done uploading piece. err: ecclient: upload failed (node:12Gbrbe3SdK7ccTugeFtmHE7gGWmLrZrGRXvb47EbiJPjfZfbKR, address:104.36.82.55:4215): protocol: rpc: dial tcp 104.36.82.55:4215: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:32:18.596890491 (57h57m1.441692015s) [noise=true , address=104.36.82.10:4092 , piece_id=RS4Z6BCUSS7VRT7G7OWPWJWG7IV3U6ERCRT35LLBO2UHN54PS53A, seg_pos={0 0}, upload=55498]: Done uploading piece. err: ecclient: upload failed (node:128P9omT47U7tm1k4XdeUR5vd1VAqskCvyfjF6VDy1ncwMDoZm9, address:104.36.82.10:4092): protocol: read tcp 159.223.143.39:41054->104.36.82.10:4092: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:32:18.596931183 (57h57m1.441732718s) [seg_pos={0 0}, upload=55498]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Jul 26 21:32:18.619416344 (57h57m1.464217878s) [upload=55498]: Done uploading segments... Jul 26 21:32:18.619451176 (57h57m1.464252712s) [upload=55498]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 1m0.026652141s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 499.491758ms FAIL (2025-07-26 21:32:18 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 499.518012ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 2.396142822s FAIL (2025-07-26 21:31:31 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.396172518s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.035914428s FAIL (2025-07-26 21:31:20 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.035933896s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 1m0.030646828s FAIL (2025-07-26 21:31:08 UTC) ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== ent... Jul 26 21:30:08.280683596 (57h54m51.12548516s) [seg_pos={0 0}, upload=55481]: Done begin upload segment. Jul 26 21:30:08.280729611 (57h54m51.125531148s) [noise=true , address=104.36.82.53:4153 , piece_id=6X3U62KA7OFGJYMEXATIKZMFVPJG262UHXLHM24VT4TDPMSBEBZA, seg_pos={0 0}, upload=55481]: Uploading piece... Jul 26 21:30:08.281245107 (57h54m51.126046667s) [noise=true , address=104.36.82.55:4228 , piece_id=CZN6N22OU3KCEA7MTRU7HSLX4WJZGSYQ3ZPGWX77WATSDYVE23LQ, seg_pos={0 0}, upload=55481]: Uploading piece... Jul 26 21:30:08.282070882 (57h54m51.126872418s) [noise=true , address=77.90.31.16:4766 , piece_id=OMJCJQNZXCHSEYWBFZ3DD45BIW5JBZB5KT2T63N6XS5JQJ43VPKA, seg_pos={0 0}, upload=55481]: Uploading piece... Jul 26 21:30:08.282713525 (57h54m51.127515062s) [noise=true , address=104.36.82.11:4130 , piece_id=S3D5YDZW4IYECJCMYQ7VVZRMCOGCNNNL7RAVD5ULO6MAUYNJWLAQ, seg_pos={0 0}, upload=55481]: Uploading piece... Jul 26 21:30:08.283202293 (57h54m51.128003829s) [noise=true , address=104.36.82.47:4541 , piece_id=RTPW5MMFI5F45IRBXC62ZP6M5Y7XFIKZIO5FSUSXS3YWJF6ZDE5A, seg_pos={0 0}, upload=55481]: Uploading piece... Jul 26 21:30:08.283579486 (57h54m51.128381026s) [noise=true , address=77.90.31.18:4826 , piece_id=CW6GCRSEKRWI4JYNTBYAO3NV7X6NZGJM6F5NMJTP42IFW3NI2FWA, seg_pos={0 0}, upload=55481]: Uploading piece... Jul 26 21:30:08.283945076 (57h54m51.128746611s) [noise=true , address=104.36.82.46:4509 , piece_id=UCT2KO5XO4ILTGBZZQZLLUABHQA3WVFENCISEPNSIXWSZWOWZ5GQ, seg_pos={0 0}, upload=55481]: Uploading piece... Jul 26 21:30:08.284956899 (57h54m51.129758444s) [noise=true , address=77.90.31.15:4717 , piece_id=GLN5PZ7GRJVVLS6WH5WPEBQTPZ2ERVGUL4Z4WZ5XJN3PEYFVZKMQ, seg_pos={0 0}, upload=55481]: Uploading piece... Jul 26 21:30:08.286117843 (57h54m51.130919379s) [noise=true , address=104.36.82.52:4097 , piece_id=GVL4LYXGK2LO5EKT5Z2PJT7JKHI275ZOBKM55AX7FNCSL4P24SQA, seg_pos={0 0}, upload=55481]: Uploading piece... Jul 26 21:30:08.287477112 (57h54m51.132278647s) [noise=true , address=64.49.13.169:4713 , piece_id=H5G2EXI4LGXQDEGIUYK3I2HQMCEDJQBL7NV2NVPI76Q2K6JO6UNA, seg_pos={0 0}, upload=55481]: Uploading piece... Jul 26 21:30:08.288127200 (57h54m51.132928736s) [noise=true , address=77.90.31.11:4558 , piece_id=SUEKQAPTO3XNTTZ2PBPULPBR67Z4SF7DZ43RN7RESF6SP7PGUAFA, seg_pos={0 0}, upload=55481]: Uploading piece... Jul 26 21:30:08.288561821 (57h54m51.133363357s) [noise=true , address=104.36.82.54:4162 , piece_id=IHNHDQOELGEHZ7YUW2RCTUJRKCYLEIFYVKLJHMG2UUF6DS7O4FFA, seg_pos={0 0}, upload=55481]: Uploading piece... Jul 26 21:30:08.288985851 (57h54m51.133787387s) [noise=true , address=104.36.82.10:4108 , piece_id=7BE4576OYWLBMF46FFGFIOMK2JYDP7NK7AX2BNLIMR3R5IBUV75Q, seg_pos={0 0}, upload=55481]: Uploading piece... Jul 26 21:30:08.289403542 (57h54m51.134205077s) [noise=true , address=156.146.43.225:4434 , piece_id=FUIG3TRBIC4XWO7DLHFG654A46HUYKLF6M6GIO3GNVE6IP4RYUCA, seg_pos={0 0}, upload=55481]: Uploading piece... Jul 26 21:30:08.289877331 (57h54m51.134678864s) [noise=true , address=79.127.201.215:4355 , piece_id=3QBG3K6GXI2DNJVJBOEBPKTKXDZXTCCGPTDD25JOILNKON55MH4Q, seg_pos={0 0}, upload=55481]: Uploading piece... Jul 26 21:30:08.290246821 (57h54m51.13506118s) [noise=true , address=79.127.219.38:4010 , piece_id=4NPI4VP7WLGM76MN5LBUXVX5PGNUFNUN3BHN3HNJLC3YKHK4L2HQ, seg_pos={0 0}, upload=55481]: Uploading piece... Jul 26 21:30:08.290685951 (57h54m51.135487487s) [noise=true , address=79.127.205.235:4680 , piece_id=5YJLMVORMWEQQKMG6OUB7N6DVX2WO5N6YSUP4BNIYF47YKPKAMSA, seg_pos={0 0}, upload=55481]: Uploading piece... Jul 26 21:30:08.290990033 (57h54m51.135791557s) [noise=true , address=104.36.82.36:4640 , piece_id=LBULUEWI6ZJ43K6PDRA56OIIZKEWV3NUORTQMCCPEKXX3ROSENZA, seg_pos={0 0}, upload=55481]: Uploading piece... Jul 26 21:30:08.521609188 (57h54m51.366410733s) [upload=55481]: Next returned nil segment Jul 26 21:30:08.521635935 (57h54m51.36643747s) [upload=55481]: Waiting for error group managing segments... Jul 26 21:30:08.559405234 (57h54m51.404206749s) [noise=true , address=104.36.82.10:4108 , piece_id=7BE4576OYWLBMF46FFGFIOMK2JYDP7NK7AX2BNLIMR3R5IBUV75Q, seg_pos={0 0}, upload=55481]: Done uploading piece. err: <nil> Jul 26 21:30:08.626797849 (57h54m51.471599379s) [noise=true , address=79.127.219.38:4010 , piece_id=4NPI4VP7WLGM76MN5LBUXVX5PGNUFNUN3BHN3HNJLC3YKHK4L2HQ, seg_pos={0 0}, upload=55481]: Done uploading piece. err: <nil> Jul 26 21:30:08.638450998 (57h54m51.483252507s) [noise=true , address=156.146.43.225:4434 , piece_id=FUIG3TRBIC4XWO7DLHFG654A46HUYKLF6M6GIO3GNVE6IP4RYUCA, seg_pos={0 0}, upload=55481]: Done uploading piece. err: <nil> Jul 26 21:30:08.970407204 (57h54m51.815208717s) [noise=true , address=79.127.205.235:4680 , piece_id=5YJLMVORMWEQQKMG6OUB7N6DVX2WO5N6YSUP4BNIYF47YKPKAMSA, seg_pos={0 0}, upload=55481]: Done uploading piece. err: <nil> Jul 26 21:30:09.311005353 (57h54m52.15580688s) [noise=true , address=79.127.201.215:4355 , piece_id=3QBG3K6GXI2DNJVJBOEBPKTKXDZXTCCGPTDD25JOILNKON55MH4Q, seg_pos={0 0}, upload=55481]: Done uploading piece. err: <nil> Jul 26 21:30:10.264544039 (57h54m53.109345575s) [noise=true , address=77.90.31.11:4558 , piece_id=SUEKQAPTO3XNTTZ2PBPULPBR67Z4SF7DZ43RN7RESF6SP7PGUAFA, seg_pos={0 0}, upload=55481]: Done uploading piece. err: <nil> Jul 26 21:30:10.719426114 (57h54m53.564227642s) [noise=true , address=77.90.31.16:4766 , piece_id=OMJCJQNZXCHSEYWBFZ3DD45BIW5JBZB5KT2T63N6XS5JQJ43VPKA, seg_pos={0 0}, upload=55481]: Done uploading piece. err: <nil> Jul 26 21:30:10.916858254 (57h54m53.76165978s) [noise=true , address=64.49.13.169:4713 , piece_id=H5G2EXI4LGXQDEGIUYK3I2HQMCEDJQBL7NV2NVPI76Q2K6JO6UNA, seg_pos={0 0}, upload=55481]: Done uploading piece. err: <nil> Jul 26 21:30:11.232693924 (57h54m54.077495439s) [noise=true , address=77.90.31.15:4717 , piece_id=GLN5PZ7GRJVVLS6WH5WPEBQTPZ2ERVGUL4Z4WZ5XJN3PEYFVZKMQ, seg_pos={0 0}, upload=55481]: Done uploading piece. err: <nil> Jul 26 21:30:12.947880261 (57h54m55.79268178s) [noise=true , address=77.90.31.18:4826 , piece_id=CW6GCRSEKRWI4JYNTBYAO3NV7X6NZGJM6F5NMJTP42IFW3NI2FWA, seg_pos={0 0}, upload=55481]: Done uploading piece. err: <nil> Jul 26 21:30:22.835990607 (57h55m5.680792136s) [noise=true , address=104.36.82.11:4130 , piece_id=S3D5YDZW4IYECJCMYQ7VVZRMCOGCNNNL7RAVD5ULO6MAUYNJWLAQ, seg_pos={0 0}, upload=55481]: Done uploading piece. err: <nil> Jul 26 21:31:08.195765686 (57h55m51.04056721s) [noise=true , address=104.36.82.55:4228 , piece_id=CZN6N22OU3KCEA7MTRU7HSLX4WJZGSYQ3ZPGWX77WATSDYVE23LQ, seg_pos={0 0}, upload=55481]: Done uploading piece. err: ecclient: upload failed (node:12NBsd33JtAwPdcbD8vBzZUYpjiqZpRzs8y2QrYJFE245pd8T5X, address:104.36.82.55:4228): protocol: rpc: dial tcp 104.36.82.55:4228: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:31:08.195784901 (57h55m51.040586429s) [noise=true , address=104.36.82.36:4640 , piece_id=LBULUEWI6ZJ43K6PDRA56OIIZKEWV3NUORTQMCCPEKXX3ROSENZA, seg_pos={0 0}, upload=55481]: Done uploading piece. err: ecclient: upload failed (node:1cL33G9zW6AsBkSfbrYC5DH1hk3ffBoNnt6CoeEy1D8SKS2hhS, address:104.36.82.36:4640): protocol: rpc: dial tcp 104.36.82.36:4640: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:31:08.195859879 (57h55m51.040661407s) [noise=true , address=104.36.82.47:4541 , piece_id=RTPW5MMFI5F45IRBXC62ZP6M5Y7XFIKZIO5FSUSXS3YWJF6ZDE5A, seg_pos={0 0}, upload=55481]: Done uploading piece. err: ecclient: upload failed (node:1m1ED9th2QhuoxBKEo3Dync9GpkWjGYNTSBHd57ud2BJoH3HXC, address:104.36.82.47:4541): protocol: write tcp 159.223.143.39:40128->104.36.82.47:4541: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:31:08.195892658 (57h55m51.040694182s) [noise=true , address=104.36.82.53:4153 , piece_id=6X3U62KA7OFGJYMEXATIKZMFVPJG262UHXLHM24VT4TDPMSBEBZA, seg_pos={0 0}, upload=55481]: Done uploading piece. err: ecclient: upload failed (node:1fPp4DvLe18dZWPmMPFiC8o9jL4uiEFNmPAHHmFEH74ZXNndw3, address:104.36.82.53:4153): protocol: rpc: dial tcp 104.36.82.53:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:31:08.195936368 (57h55m51.040737896s) [noise=true , address=104.36.82.54:4162 , piece_id=IHNHDQOELGEHZ7YUW2RCTUJRKCYLEIFYVKLJHMG2UUF6DS7O4FFA, seg_pos={0 0}, upload=55481]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:104.36.82.54:4162): protocol: rpc: dial tcp 104.36.82.54:4162: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:31:08.195984108 (57h55m51.040785631s) [noise=true , address=104.36.82.52:4097 , piece_id=GVL4LYXGK2LO5EKT5Z2PJT7JKHI275ZOBKM55AX7FNCSL4P24SQA, seg_pos={0 0}, upload=55481]: Done uploading piece. err: ecclient: upload failed (node:1sMhTktqf2H4aZSyFV3m9Z6eMVxL2U52bvUHVHUh2gUCVp32AM, address:104.36.82.52:4097): protocol: write tcp 159.223.143.39:52470->104.36.82.52:4097: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:31:08.196027629 (57h55m51.040829145s) [noise=true , address=104.36.82.46:4509 , piece_id=UCT2KO5XO4ILTGBZZQZLLUABHQA3WVFENCISEPNSIXWSZWOWZ5GQ, seg_pos={0 0}, upload=55481]: Done uploading piece. err: ecclient: upload failed (node:1y2NmauNvCyrMJ5z66Lihb3h3C9nwdJYqfSfNmC5dwDGYQqUKh, address:104.36.82.46:4509): protocol: write tcp 159.223.143.39:36848->104.36.82.46:4509: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:31:08.196070677 (57h55m51.040872208s) [seg_pos={0 0}, upload=55481]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Jul 26 21:31:08.224466625 (57h55m51.069268157s) [upload=55481]: Done uploading segments... Jul 26 21:31:08.224499785 (57h55m51.069301318s) [upload=55481]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 1m0.03071569s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 1m0.035856603s FAIL (2025-07-26 21:30:05 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== UU33327HMD5TTMQHADH5NG22FQQ, seg_pos={0 0}, upload=55462]: Uploading piece... Jul 26 21:29:05.775329025 (57h53m48.620130559s) [noise=true , address=104.36.82.47:4549 , piece_id=E6IRXMSY7PB5ERJSACBR4KYVF4UXIGIMTNUJFPVJCQYL7FO2NIHQ, seg_pos={0 0}, upload=55462]: Uploading piece... Jul 26 21:29:05.775933903 (57h53m48.620735434s) [noise=true , address=104.36.82.46:4510 , piece_id=MLGTKN2UKJRUVJ77L6TW6SQW73JA5AEYO6B2DW2RVPJVOVCJBXPA, seg_pos={0 0}, upload=55462]: Uploading piece... Jul 26 21:29:05.776549071 (57h53m48.621350608s) [noise=true , address=104.36.82.35:4588 , piece_id=RVTH2A7BR3FN6QEI66ZLK6LSEHHFK6SUWSGJGF5H4JA3CLRZ44VA, seg_pos={0 0}, upload=55462]: Uploading piece... Jul 26 21:29:05.779621607 (57h53m48.624423141s) [noise=true , address=79.127.201.215:4343 , piece_id=WWIU7D3FBON3T2VOFLPJATN4GHI6772YTWGQEDBQ4CEMMCLACJMQ, seg_pos={0 0}, upload=55462]: Uploading piece... Jul 26 21:29:05.780157718 (57h53m48.624959253s) [noise=true , address=104.36.82.51:4081 , piece_id=VPXBKKFE5YCS32LSTKE2DLOYPZGYATMAOU2LJG3ZPFTXV3GGCXSQ, seg_pos={0 0}, upload=55462]: Uploading piece... Jul 26 21:29:05.780641477 (57h53m48.625443013s) [noise=true , address=104.36.82.55:4205 , piece_id=MCIM3PUSHRQJOOW7FRO76FSFLJC4UT54AGHU4PTDLC7ZRUWSSHSA, seg_pos={0 0}, upload=55462]: Uploading piece... Jul 26 21:29:05.781106484 (57h53m48.625908017s) [noise=true , address=104.36.82.42:4798 , piece_id=W4NGTX4KFY2EGF2YIGCZ2HHSR5I6RZNO3VHKY7IXHHQAHSK4H6LQ, seg_pos={0 0}, upload=55462]: Uploading piece... Jul 26 21:29:05.781576604 (57h53m48.626378139s) [noise=true , address=104.36.82.52:4111 , piece_id=D5Y322NZDK536QHKR7FZFRBFOFXYPI7ON3272T5FE6OFQ2VKS5ZQ, seg_pos={0 0}, upload=55462]: Uploading piece... Jul 26 21:29:05.784027535 (57h53m48.628829072s) [noise=true , address=79.127.205.235:4691 , piece_id=XW3BVUXDG7EAMPAPLYK23CVJJC7S2VKRIXAYBHKFKZP3HGWF7CLQ, seg_pos={0 0}, upload=55462]: Uploading piece... Jul 26 21:29:05.786289355 (57h53m48.6310909s) [noise=true , address=79.127.219.38:4003 , piece_id=5CITVTFIZO3PKWJVVMYRSA76LZJLVFLV4RNDJSKVRWNADL5CAVVQ, seg_pos={0 0}, upload=55462]: Uploading piece... Jul 26 21:29:05.787735224 (57h53m48.632536758s) [noise=true , address=104.36.82.53:4128 , piece_id=XHI6ZM24KMEKEBEF4SNEOIB3PT4AZUEYV5N24QYMPDQEGLQUSGRQ, seg_pos={0 0}, upload=55462]: Uploading piece... Jul 26 21:29:05.791645772 (57h53m48.636447306s) [noise=true , address=104.36.82.56:4263 , piece_id=AG6GZDZG7UJ67V4IPQHRUPX3E5NEFTDSIYVQ6VKGJ53QJE3N6C4Q, seg_pos={0 0}, upload=55462]: Uploading piece... Jul 26 21:29:05.911979105 (57h53m48.756780652s) [upload=55462]: Next returned nil segment Jul 26 21:29:05.912003919 (57h53m48.756805456s) [upload=55462]: Waiting for error group managing segments... Jul 26 21:29:05.921941563 (57h53m48.766743081s) [noise=true , address=79.127.219.38:4009 , piece_id=AZY4EZOLD6ANGLUJQADUQ7O4CMOQRPYEQX547SDUNIIIJV55OZDA, seg_pos={0 0}, upload=55462]: Done uploading piece. err: <nil> Jul 26 21:29:06.330490815 (57h53m49.17529239s) [noise=true , address=79.127.201.215:4343 , piece_id=WWIU7D3FBON3T2VOFLPJATN4GHI6772YTWGQEDBQ4CEMMCLACJMQ, seg_pos={0 0}, upload=55462]: Done uploading piece. err: <nil> Jul 26 21:29:06.414404049 (57h53m49.259205559s) [noise=true , address=79.127.205.235:4691 , piece_id=XW3BVUXDG7EAMPAPLYK23CVJJC7S2VKRIXAYBHKFKZP3HGWF7CLQ, seg_pos={0 0}, upload=55462]: Done uploading piece. err: <nil> Jul 26 21:29:06.617407537 (57h53m49.462209061s) [noise=true , address=79.127.205.245:4231 , piece_id=CLGEXZ5SAEOQSFQVTK2FLNC4NVAIVJQ4CLIO2LTPK2TZFLS5JBHA, seg_pos={0 0}, upload=55462]: Done uploading piece. err: <nil> Jul 26 21:29:08.145035381 (57h53m50.989836917s) [noise=true , address=79.127.219.38:4003 , piece_id=5CITVTFIZO3PKWJVVMYRSA76LZJLVFLV4RNDJSKVRWNADL5CAVVQ, seg_pos={0 0}, upload=55462]: Done uploading piece. err: <nil> Jul 26 21:29:08.710418402 (57h53m51.55521992s) [noise=true , address=77.90.31.11:4574 , piece_id=CDWB6E3IDEKT6OT4DCXL6IUP7S2VPRAU6FPUGFVH3MAPMG5TPIHA, seg_pos={0 0}, upload=55462]: Done uploading piece. err: <nil> Jul 26 21:30:05.696287670 (57h54m48.541089198s) [noise=true , address=104.36.82.46:4510 , piece_id=MLGTKN2UKJRUVJ77L6TW6SQW73JA5AEYO6B2DW2RVPJVOVCJBXPA, seg_pos={0 0}, upload=55462]: Done uploading piece. err: ecclient: upload failed (node:1c6o4CCFBx5xgnmaivGFzRGqqp8qrqwyzwTrcpwDYFh71wRQSZ, address:104.36.82.46:4510): protocol: rpc: dial tcp 104.36.82.46:4510: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:30:05.696518371 (57h54m48.541319905s) [noise=true , address=104.36.82.53:4128 , piece_id=XHI6ZM24KMEKEBEF4SNEOIB3PT4AZUEYV5N24QYMPDQEGLQUSGRQ, seg_pos={0 0}, upload=55462]: Done uploading piece. err: ecclient: upload failed (node:125ZMc6ed9qgBRtBC5N4zNvGonip3GNAEBPt3auF6dBGteX46Pf, address:104.36.82.53:4128): protocol: write tcp 159.223.143.39:38684->104.36.82.53:4128: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:30:05.696590294 (57h54m48.541391824s) [noise=true , address=104.36.82.47:4549 , piece_id=E6IRXMSY7PB5ERJSACBR4KYVF4UXIGIMTNUJFPVJCQYL7FO2NIHQ, seg_pos={0 0}, upload=55462]: Done uploading piece. err: ecclient: upload failed (node:1HWoyMKQ6v2XWUfEmfFh3m8AiGivAcYkkZ9na8kqHFzH2hPDeK, address:104.36.82.47:4549): protocol: rpc: dial tcp 104.36.82.47:4549: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:30:05.696651928 (57h54m48.541453454s) [noise=true , address=104.36.82.35:4588 , piece_id=RVTH2A7BR3FN6QEI66ZLK6LSEHHFK6SUWSGJGF5H4JA3CLRZ44VA, seg_pos={0 0}, upload=55462]: Done uploading piece. err: ecclient: upload failed (node:1nMFTs5XGAKT6WGe7BRrAPNvuE1oCN7ZfiCfvWWwGDkCsKmQo5, address:104.36.82.35:4588): protocol: write tcp 159.223.143.39:34690->104.36.82.35:4588: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:30:05.696801676 (57h54m48.541603207s) [noise=true , address=104.36.82.56:4263 , piece_id=AG6GZDZG7UJ67V4IPQHRUPX3E5NEFTDSIYVQ6VKGJ53QJE3N6C4Q, seg_pos={0 0}, upload=55462]: Done uploading piece. err: ecclient: upload failed (node:12tLW4r8AQKLCbSWZoNUVi5v2YtF8NthUmX81TLQn8G8TotuByG, address:104.36.82.56:4263): protocol: write tcp 159.223.143.39:52602->104.36.82.56:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:30:05.696860743 (57h54m48.541662274s) [noise=true , address=104.36.82.7:4940 , piece_id=PKGRYCTKMU4K2DJDUCLHNOP5UNUJXWQGMNN4NP5HXAMDYJWJF7GQ, seg_pos={0 0}, upload=55462]: Done uploading piece. err: ecclient: upload failed (node:12mPAwJqTNf7V3U291WVZNzMf3Tzi1fb4cQiJUEB8RH99xN13n, address:104.36.82.7:4940): protocol: write tcp 159.223.143.39:58698->104.36.82.7:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:30:05.696923288 (57h54m48.541724821s) [noise=true , address=104.36.82.34:4573 , piece_id=FH77BY2YJ2HGZREF5JQHV2554TUHVSTLX7XKLXIKKPBB4ECTRPTA, seg_pos={0 0}, upload=55462]: Done uploading piece. err: ecclient: upload failed (node:1yD3c1tEtGt2yYEyzDX5GugUChtkCQa9VTWkBH9ftgwvqn72SU, address:104.36.82.34:4573): protocol: write tcp 159.223.143.39:34600->104.36.82.34:4573: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:30:05.696982512 (57h54m48.541784042s) [noise=true , address=104.36.82.47:4527 , piece_id=KRBWJWFT5CXAGWKZ5KZDS3OQOUU33327HMD5TTMQHADH5NG22FQQ, seg_pos={0 0}, upload=55462]: Done uploading piece. err: ecclient: upload failed (node:17xmkbFoj3PTJnAygsjRAcfdd17AfSkPP2xdZp68GGPrvfMuU6, address:104.36.82.47:4527): protocol: rpc: dial tcp 104.36.82.47:4527: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:30:05.697046157 (57h54m48.541847688s) [noise=true , address=104.36.82.55:4205 , piece_id=MCIM3PUSHRQJOOW7FRO76FSFLJC4UT54AGHU4PTDLC7ZRUWSSHSA, seg_pos={0 0}, upload=55462]: Done uploading piece. err: ecclient: upload failed (node:12UafxBVo9nuRVBcQmGE4gBP2pnXQ5YUrwHZdwPCeSBSBWCZKpQ, address:104.36.82.55:4205): protocol: write tcp 159.223.143.39:44208->104.36.82.55:4205: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:30:05.697144560 (57h54m48.541946087s) [noise=true , address=104.36.82.52:4111 , piece_id=D5Y322NZDK536QHKR7FZFRBFOFXYPI7ON3272T5FE6OFQ2VKS5ZQ, seg_pos={0 0}, upload=55462]: Done uploading piece. err: ecclient: upload failed (node:12krqZcFoBPfLFbLGQUAjPMzHJ4HKqsKPRg6Xap7nuShaEKBumH, address:104.36.82.52:4111): protocol: rpc: dial tcp 104.36.82.52:4111: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:30:05.697241040 (57h54m48.542042569s) [noise=true , address=104.36.82.51:4081 , piece_id=VPXBKKFE5YCS32LSTKE2DLOYPZGYATMAOU2LJG3ZPFTXV3GGCXSQ, seg_pos={0 0}, upload=55462]: Done uploading piece. err: ecclient: upload failed (node:121Gj3trM89mKpTffMYkrV9PrwkPqQU2LauCiG1v3J91Mej9KWF, address:104.36.82.51:4081): protocol: write tcp 159.223.143.39:60648->104.36.82.51:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:30:05.697296414 (57h54m48.542097944s) [noise=true , address=104.36.82.42:4798 , piece_id=W4NGTX4KFY2EGF2YIGCZ2HHSR5I6RZNO3VHKY7IXHHQAHSK4H6LQ, seg_pos={0 0}, upload=55462]: Done uploading piece. err: ecclient: upload failed (node:17Ski7oMzbGroPGgWRrbUiqC1NiLPBBsvYuU4pCZ53xhNKkeBZ, address:104.36.82.42:4798): protocol: write tcp 159.223.143.39:44490->104.36.82.42:4798: use of closed network connection; conn closed: op Jul 26 21:30:05.697332899 (57h54m48.54213443s) [seg_pos={0 0}, upload=55462]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Jul 26 21:30:05.724336634 (57h54m48.569138162s) [upload=55462]: Done uploading segments... Jul 26 21:30:05.724409956 (57h54m48.569211488s) [upload=55462]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 1m0.035875659s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.010727753s FAIL (2025-07-26 21:29:05 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : false Jul 26 21:28:05.610264476 (57h52m48.455066002s) [upload=55448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:28:05.610272282 (57h52m48.455073811s) [upload=55448]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:28:05.647427679 (57h52m48.492229201s) [seg_pos={0 0}, upload=55448]: Begin upload segment... Jul 26 21:28:05.647517110 (57h52m48.492318635s) [seg_pos={0 0}, upload=55448]: Done begin upload segment. Jul 26 21:28:05.647548988 (57h52m48.492350519s) [noise=true , address=64.49.13.138:4482 , piece_id=KLS3NSY7L3GHXUVL5JKSOYPEMHW6FM5TUYQBOZJYBU7B2AA7IIDA, seg_pos={0 0}, upload=55448]: Uploading piece... Jul 26 21:28:05.647560175 (57h52m48.492361714s) [noise=true , address=104.36.82.55:4212 , piece_id=HB6VV7WTXZMZP2HWMXA3ZYGHXSMOKK6JRFESNX4LLYKCLWZ4KGHQ, seg_pos={0 0}, upload=55448]: Uploading piece... Jul 26 21:28:05.648012396 (57h52m48.492813921s) [noise=true , address=104.36.82.37:4681 , piece_id=ABFXF3Q4DV4DM2LQLSUQF4U42TEYDODBCQ6GEIHVXRDB42Y5Z2XQ, seg_pos={0 0}, upload=55448]: Uploading piece... Jul 26 21:28:05.648095416 (57h52m48.492896947s) [noise=true , address=104.36.82.46:4493 , piece_id=MFXWE4KJN3OH4XNF4R2ZNZLUXGT7NTBR24X5DDF3VVHRM2XVA4XA, seg_pos={0 0}, upload=55448]: Uploading piece... Jul 26 21:28:05.648418555 (57h52m48.493220094s) [noise=true , address=104.36.82.9:4035 , piece_id=F3NNF3XNH2Z5CMX7ZFAN6YEQPVTM5L3ZYEDTXTOJIPS5OVTOXAQQ, seg_pos={0 0}, upload=55448]: Uploading piece... Jul 26 21:28:05.648734482 (57h52m48.493536005s) [noise=true , address=79.127.205.234:4612 , piece_id=DCUYBRBJCYQTRBIIYCXEYEIXAYNCBKHLXTOMTLPZMJIV7R6W4C7Q, seg_pos={0 0}, upload=55448]: Uploading piece... Jul 26 21:28:05.648780931 (57h52m48.493582454s) [noise=true , address=104.36.82.3:4785 , piece_id=RWEQGP3CK6SRR3POV3IFEBB6XFKVVKTJQSV6S32J5SDQCV2Y6SRQ, seg_pos={0 0}, upload=55448]: Uploading piece... Jul 26 21:28:05.655580578 (57h52m48.500382103s) [noise=true , address=104.36.82.47:4536 , piece_id=PWASPRPOUETD2AD3SEF3MR5XI3QW4DG72CZ2R2EM2DV5JL2MURAQ, seg_pos={0 0}, upload=55448]: Uploading piece... Jul 26 21:28:05.656742407 (57h52m48.501543941s) [noise=true , address=104.36.82.52:4119 , piece_id=7TJ5654TC5BUFMIWWGVHDZ776E5HFOGMYJCOUYW2INXDTFVDKRKQ, seg_pos={0 0}, upload=55448]: Uploading piece... Jul 26 21:28:05.657662689 (57h52m48.502464215s) [noise=true , address=79.127.205.245:4236 , piece_id=P6MQ4KCONYLLEW5EGXC7A5ZMLXT62S2JQG5QEE7IUZ3KA5WM4A4Q, seg_pos={0 0}, upload=55448]: Uploading piece... Jul 26 21:28:05.658324283 (57h52m48.503125809s) [noise=true , address=104.36.82.20:4467 , piece_id=LKAA3JQ7AH2ENXPWBDPGUJKTKEVMZDGUBA3H3MDN2OW4Y2Y5T3YA, seg_pos={0 0}, upload=55448]: Uploading piece... Jul 26 21:28:05.659008602 (57h52m48.503810127s) [noise=true , address=104.36.82.54:4161 , piece_id=A5NZQDF6UYKSL3J7PQXUW6PNKA554N5V62MF4SKFQMHYPL3IVKLQ, seg_pos={0 0}, upload=55448]: Uploading piece... Jul 26 21:28:05.659505570 (57h52m48.504307093s) [noise=true , address=77.90.31.13:4656 , piece_id=ZFZTJM6F76EGJF6JZM4WSMBYVQN67LUWJO3KDUDCHYE3XMFKEZLQ, seg_pos={0 0}, upload=55448]: Uploading piece... Jul 26 21:28:05.659876357 (57h52m48.504677882s) [noise=true , address=104.36.82.53:4154 , piece_id=VYCXYSO25JBMP6AMLBX2H73XXVHXARPM6WU7PTQYDRRO6KFVYSCA, seg_pos={0 0}, upload=55448]: Uploading piece... Jul 26 21:28:05.660374428 (57h52m48.505175958s) [noise=true , address=104.36.82.34:4578 , piece_id=WNPQ4KG4CQOCIS2RYTR5M5UXOFIMI2NXTWX4TG22MM33POWURAKA, seg_pos={0 0}, upload=55448]: Uploading piece... Jul 26 21:28:05.660848018 (57h52m48.505649541s) [noise=true , address=79.127.205.235:4678 , piece_id=LJP6MWE2LUHL7MT5BKZPV4IFP5NUH3UG6RIJEYDKFO447SU66IGQ, seg_pos={0 0}, upload=55448]: Uploading piece... Jul 26 21:28:05.661169040 (57h52m48.505970562s) [noise=true , address=64.49.13.140:4514 , piece_id=YPYX7V2X6UT7XV7SJ5CQMGTLPNFHQZA5LF7UE4FGBPJFFGW7Y2JA, seg_pos={0 0}, upload=55448]: Uploading piece... Jul 26 21:28:05.661438873 (57h52m48.506240398s) [noise=true , address=104.36.82.56:4257 , piece_id=7OZAQI2PEDCLQA76Y7PF73DHRQNXVEVCK6UJTZYGAQIA7L2OI7IQ, seg_pos={0 0}, upload=55448]: Uploading piece... Jul 26 21:28:05.757005289 (57h52m48.60180682s) [upload=55448]: Next returned nil segment Jul 26 21:28:05.757030489 (57h52m48.601832023s) [upload=55448]: Waiting for error group managing segments... Jul 26 21:28:05.777141748 (57h52m48.621943275s) [noise=true , address=104.36.82.54:4161 , piece_id=A5NZQDF6UYKSL3J7PQXUW6PNKA554N5V62MF4SKFQMHYPL3IVKLQ, seg_pos={0 0}, upload=55448]: Done uploading piece. err: <nil> Jul 26 21:28:05.815927243 (57h52m48.660728752s) [noise=true , address=104.36.82.55:4212 , piece_id=HB6VV7WTXZMZP2HWMXA3ZYGHXSMOKK6JRFESNX4LLYKCLWZ4KGHQ, seg_pos={0 0}, upload=55448]: Done uploading piece. err: <nil> Jul 26 21:28:06.017876035 (57h52m48.862677562s) [noise=true , address=77.90.31.13:4656 , piece_id=ZFZTJM6F76EGJF6JZM4WSMBYVQN67LUWJO3KDUDCHYE3XMFKEZLQ, seg_pos={0 0}, upload=55448]: Done uploading piece. err: <nil> Jul 26 21:28:06.160776159 (57h52m49.005577682s) [noise=true , address=104.36.82.52:4119 , piece_id=7TJ5654TC5BUFMIWWGVHDZ776E5HFOGMYJCOUYW2INXDTFVDKRKQ, seg_pos={0 0}, upload=55448]: Done uploading piece. err: <nil> Jul 26 21:28:06.249641501 (57h52m49.094443026s) [noise=true , address=79.127.205.245:4236 , piece_id=P6MQ4KCONYLLEW5EGXC7A5ZMLXT62S2JQG5QEE7IUZ3KA5WM4A4Q, seg_pos={0 0}, upload=55448]: Done uploading piece. err: <nil> Jul 26 21:28:06.276257199 (57h52m49.121058718s) [noise=true , address=79.127.205.234:4612 , piece_id=DCUYBRBJCYQTRBIIYCXEYEIXAYNCBKHLXTOMTLPZMJIV7R6W4C7Q, seg_pos={0 0}, upload=55448]: Done uploading piece. err: <nil> Jul 26 21:28:06.287391113 (57h52m49.132192642s) [noise=true , address=79.127.205.235:4678 , piece_id=LJP6MWE2LUHL7MT5BKZPV4IFP5NUH3UG6RIJEYDKFO447SU66IGQ, seg_pos={0 0}, upload=55448]: Done uploading piece. err: <nil> Jul 26 21:28:06.314015525 (57h52m49.15881706s) [noise=true , address=64.49.13.138:4482 , piece_id=KLS3NSY7L3GHXUVL5JKSOYPEMHW6FM5TUYQBOZJYBU7B2AA7IIDA, seg_pos={0 0}, upload=55448]: Done uploading piece. err: <nil> Jul 26 21:28:06.356994268 (57h52m49.201795777s) [noise=true , address=64.49.13.140:4514 , piece_id=YPYX7V2X6UT7XV7SJ5CQMGTLPNFHQZA5LF7UE4FGBPJFFGW7Y2JA, seg_pos={0 0}, upload=55448]: Done uploading piece. err: <nil> Jul 26 21:28:07.281854820 (57h52m50.126656352s) [noise=true , address=104.36.82.53:4154 , piece_id=VYCXYSO25JBMP6AMLBX2H73XXVHXARPM6WU7PTQYDRRO6KFVYSCA, seg_pos={0 0}, upload=55448]: Done uploading piece. err: <nil> Jul 26 21:28:07.387545946 (57h52m50.232347461s) [noise=true , address=104.36.82.9:4035 , piece_id=F3NNF3XNH2Z5CMX7ZFAN6YEQPVTM5L3ZYEDTXTOJIPS5OVTOXAQQ, seg_pos={0 0}, upload=55448]: Done uploading piece. err: <nil> Jul 26 21:28:07.601327873 (57h52m50.446129405s) [noise=true , address=104.36.82.47:4536 , piece_id=PWASPRPOUETD2AD3SEF3MR5XI3QW4DG72CZ2R2EM2DV5JL2MURAQ, seg_pos={0 0}, upload=55448]: Done uploading piece. err: <nil> Jul 26 21:28:09.233467890 (57h52m52.078269418s) [noise=true , address=104.36.82.37:4681 , piece_id=ABFXF3Q4DV4DM2LQLSUQF4U42TEYDODBCQ6GEIHVXRDB42Y5Z2XQ, seg_pos={0 0}, upload=55448]: Done uploading piece. err: <nil> Jul 26 21:28:09.279231934 (57h52m52.12403346s) [noise=true , address=104.36.82.34:4578 , piece_id=WNPQ4KG4CQOCIS2RYTR5M5UXOFIMI2NXTWX4TG22MM33POWURAKA, seg_pos={0 0}, upload=55448]: Done uploading piece. err: <nil> Jul 26 21:28:09.279286244 (57h52m52.124087773s) [seg_pos={0 0}, upload=55448]: Segment reached optimal threshold of 14 pieces. Jul 26 21:28:09.280943659 (57h52m52.125745193s) [noise=true , address=104.36.82.46:4493 , piece_id=MFXWE4KJN3OH4XNF4R2ZNZLUXGT7NTBR24X5DDF3VVHRM2XVA4XA, seg_pos={0 0}, upload=55448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127drPYZWm7RRXuCLtuzDgSaxxuu3k1tg6eo3KWtrQGjBaUbfRJ): protocol: rpc: dial tcp 104.36.82.46:4493: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:28:09.281202326 (57h52m52.126003855s) [noise=true , address=104.36.82.3:4785 , piece_id=RWEQGP3CK6SRR3POV3IFEBB6XFKVVKTJQSV6S32J5SDQCV2Y6SRQ, seg_pos={0 0}, upload=55448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u5zY7F7oy2tPUyFfoiebzFEySetBks9LC3gRGPURAyDDnsUu9): protocol: write tcp 159.223.143.39:35102->104.36.82.3:4785: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:28:09.281399401 (57h52m52.126200934s) [noise=true , address=104.36.82.56:4257 , piece_id=7OZAQI2PEDCLQA76Y7PF73DHRQNXVEVCK6UJTZYGAQIA7L2OI7IQ, seg_pos={0 0}, upload=55448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12iw1KEPBAYhvSachvkigv7bPXfGN5XEugvxGC5YTBV2Muymir3): protocol: read tcp 159.223.143.39:53890->104.36.82.56:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:28:09.281469012 (57h52m52.126270523s) [noise=true , address=104.36.82.20:4467 , piece_id=LKAA3JQ7AH2ENXPWBDPGUJKTKEVMZDGUBA3H3MDN2OW4Y2Y5T3YA, seg_pos={0 0}, upload=55448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f6rDHMRpZZmvJxShMhAS5KKGbEEgNaBayPKCyxVqpPk4knnbR): protocol: rpc: dial tcp 104.36.82.20:4467: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:28:09.281504714 (57h52m52.126306246s) [seg_pos={0 0}, upload=55448]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 26 21:28:09.281639383 (57h52m52.126440916s) [upload=55448]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 21:28:09.281664490 (57h52m52.126466026s) [upload=55448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 21:28:09.464020515 (57h52m52.308822047s) [upload=55448]: Done uploading segments... Jul 26 21:28:09.464044913 (57h52m52.30884644s) [upload=55448]: Upload finished. err: <nil> Duration: 1m0.010790378s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 4.124963564s FAIL (2025-07-26 21:14:15 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== :22.446786005 (56h53m5.29158752s) [upload=54498]: Done starting upload Jul 26 20:28:22.446976619 (56h53m5.291778131s) [upload=54498]: Uploading segments... Jul 26 20:28:22.447194509 (56h53m5.291996032s) [upload=54498]: Got next segment. Inline: false Jul 26 20:28:22.447258557 (56h53m5.292060069s) [upload=54498]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:22.447264559 (56h53m5.292066138s) [upload=54498]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:22.481429047 (56h53m5.32623056s) [seg_pos={0 0}, upload=54498]: Begin upload segment... Jul 26 20:28:22.481500645 (56h53m5.32630217s) [seg_pos={0 0}, upload=54498]: Done begin upload segment. Jul 26 20:28:22.481533046 (56h53m5.326334571s) [noise=true , address=45.91.116.4:4938 , piece_id=XPEH2CKEBJUEDAZGL4W6SZYHFCOSVP6IU2OWIU44EVU46QB7TQOA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.481538519 (56h53m5.326340049s) [noise=true , address=45.91.116.8:4047 , piece_id=QCPM7CFY5UBXWAZEAFO57NELB5R62GZ7CG2ISD2TSWAMNG6LYTRQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.481625224 (56h53m5.326426746s) [noise=true , address=77.90.31.12:4589 , piece_id=VUCKZDOASEF646TUXJEEUR2IEMVPAE4LV4I57AB6HMNQG2CZCTWA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.482010694 (56h53m5.326812226s) [noise=true , address=104.36.82.8:4969 , piece_id=EQKLKHNZE7DGZOBGZRBUA4BTVJU6AVHKHK7CZAXCEKTKULNG2E5A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.482180452 (56h53m5.326981984s) [noise=true , address=77.90.31.21:4878 , piece_id=3L4TERTQQBVRHMR2IXFGYGPVDJQK6KCPS6DFOJCI2KAVG5KBLJ3A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.482706547 (56h53m5.327508072s) [noise=true , address=77.90.31.15:4698 , piece_id=UZBDOABD2WFFPYJCC5EYJUZX6NU4REM5MRZF7POAQR3TXTZ5HXPQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.483145730 (56h53m5.327947256s) [noise=true , address=77.90.31.14:4662 , piece_id=LFCOMWII6WAEYRC3J4NMTC2RNOS3MX4WNLDUFPI4TGPWO64AFOXQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.483696003 (56h53m5.328497531s) [noise=true , address=45.91.116.12:4226 , piece_id=FY47OOHAOWN4DZCL446MZKMNEZVUFH37DEFUGOKHDAYNG3QFFRNQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.483766294 (56h53m5.328567822s) [noise=true , address=77.90.31.19:4843 , piece_id=GY3TBUVRZWDCJKIBM3GBPA2MMDPKB3VYF6CIRTWJ32MHXXNIDX7Q, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.484435977 (56h53m5.329237499s) [noise=true , address=45.91.116.13:4154 , piece_id=H6XMHU4LQO7T7NVPKAOBU3TJMJSGDS2SDZIE2KYYCXNT2MK4MFOQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.484911788 (56h53m5.329713311s) [noise=true , address=45.91.116.10:4082 , piece_id=C6G2APYQRU3LLXOZFU2PWQRXR3OQ2WYIADMJ76J2WVDY7HKNDTNA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.487491433 (56h53m5.332292958s) [noise=true , address=77.90.31.13:4626 , piece_id=KTVRUR4P2Z3N5CB26BLEIBORDOSZQTZM4D3E2IHEGZ4QXXWDBPXA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.487914119 (56h53m5.332715646s) [noise=true , address=77.90.31.21:4877 , piece_id=UHPHX5KJKOB3DGWIWTMPN7P73V76GAVCOLOO4GTPMQ6BDGTCMA4A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.488256189 (56h53m5.333057714s) [noise=true , address=45.91.116.14:4191 , piece_id=M6ZKO6X2XJ7UMFZWSGNWUIEYDTXJB6C62ZXE7SVRIF6XOMCCQD4Q, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.488930410 (56h53m5.333731944s) [noise=true , address=45.91.116.6:4974 , piece_id=3W43RVQMPBJXQJWYUYX4IJ7CZUP2SWCMXZEB4AL6ITZ3OVPMZA2A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.489313463 (56h53m5.334114987s) [noise=true , address=45.91.116.11:4119 , piece_id=DRIZSI6XSABNTXWUMNFY3RPV3HPUBVE62HXN7XG55FM44TMVCYSQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:23.003858627 (56h53m5.848660156s) [upload=54498]: Next returned nil segment Jul 26 20:28:23.003881158 (56h53m5.848682712s) [upload=54498]: Waiting for error group managing segments... Jul 26 20:28:23.117181340 (56h53m5.961982857s) [noise=true , address=77.90.31.19:4843 , piece_id=GY3TBUVRZWDCJKIBM3GBPA2MMDPKB3VYF6CIRTWJ32MHXXNIDX7Q, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.121502780 (56h53m5.966304296s) [noise=true , address=77.90.31.15:4698 , piece_id=UZBDOABD2WFFPYJCC5EYJUZX6NU4REM5MRZF7POAQR3TXTZ5HXPQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.128715466 (56h53m5.973516987s) [noise=true , address=77.90.31.12:4589 , piece_id=VUCKZDOASEF646TUXJEEUR2IEMVPAE4LV4I57AB6HMNQG2CZCTWA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.136428815 (56h53m5.981230348s) [noise=true , address=77.90.31.14:4662 , piece_id=LFCOMWII6WAEYRC3J4NMTC2RNOS3MX4WNLDUFPI4TGPWO64AFOXQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.140913043 (56h53m5.985714576s) [noise=true , address=77.90.31.13:4626 , piece_id=KTVRUR4P2Z3N5CB26BLEIBORDOSZQTZM4D3E2IHEGZ4QXXWDBPXA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.144098231 (56h53m5.988899746s) [noise=true , address=77.90.31.21:4877 , piece_id=UHPHX5KJKOB3DGWIWTMPN7P73V76GAVCOLOO4GTPMQ6BDGTCMA4A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.154118120 (56h53m5.998919628s) [noise=true , address=77.90.31.21:4878 , piece_id=3L4TERTQQBVRHMR2IXFGYGPVDJQK6KCPS6DFOJCI2KAVG5KBLJ3A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.851371196 (56h53m6.696172711s) [noise=true , address=104.36.82.8:4969 , piece_id=EQKLKHNZE7DGZOBGZRBUA4BTVJU6AVHKHK7CZAXCEKTKULNG2E5A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:52.417946210 (56h53m35.262747746s) [noise=true , address=45.91.116.8:4047 , piece_id=QCPM7CFY5UBXWAZEAFO57NELB5R62GZ7CG2ISD2TSWAMNG6LYTRQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: rpc: dial tcp 45.91.116.8:4047: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418043876 (56h53m35.26284539s) [noise=true , address=45.91.116.14:4191 , piece_id=M6ZKO6X2XJ7UMFZWSGNWUIEYDTXJB6C62ZXE7SVRIF6XOMCCQD4Q, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418090229 (56h53m35.262891746s) [noise=true , address=45.91.116.13:4154 , piece_id=H6XMHU4LQO7T7NVPKAOBU3TJMJSGDS2SDZIE2KYYCXNT2MK4MFOQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 159.223.143.39:42688->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418225695 (56h53m35.263027225s) [noise=true , address=45.91.116.10:4082 , piece_id=C6G2APYQRU3LLXOZFU2PWQRXR3OQ2WYIADMJ76J2WVDY7HKNDTNA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: write tcp 159.223.143.39:60366->45.91.116.10:4082: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418340242 (56h53m35.263141768s) [noise=true , address=45.91.116.12:4226 , piece_id=FY47OOHAOWN4DZCL446MZKMNEZVUFH37DEFUGOKHDAYNG3QFFRNQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: write tcp 159.223.143.39:37720->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418429162 (56h53m35.263230689s) [noise=true , address=45.91.116.4:4938 , piece_id=XPEH2CKEBJUEDAZGL4W6SZYHFCOSVP6IU2OWIU44EVU46QB7TQOA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418473168 (56h53m35.263274683s) [noise=true , address=45.91.116.6:4974 , piece_id=3W43RVQMPBJXQJWYUYX4IJ7CZUP2SWCMXZEB4AL6ITZ3OVPMZA2A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 159.223.143.39:37724->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418564079 (56h53m35.263365602s) [noise=true , address=45.91.116.11:4119 , piece_id=DRIZSI6XSABNTXWUMNFY3RPV3HPUBVE62HXN7XG55FM44TMVCYSQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:44972->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418628408 (56h53m35.263429934s) [seg_pos={0 0}, upload=54498]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:52.444538177 (56h53m35.289339693s) [upload=54498]: Done uploading segments... Jul 26 20:28:52.444571403 (56h53m35.289372917s) [upload=54498]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 4.125015246s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 527.024053ms FAIL (2025-07-26 21:13:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== :22.446786005 (56h53m5.29158752s) [upload=54498]: Done starting upload Jul 26 20:28:22.446976619 (56h53m5.291778131s) [upload=54498]: Uploading segments... Jul 26 20:28:22.447194509 (56h53m5.291996032s) [upload=54498]: Got next segment. Inline: false Jul 26 20:28:22.447258557 (56h53m5.292060069s) [upload=54498]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:22.447264559 (56h53m5.292066138s) [upload=54498]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:22.481429047 (56h53m5.32623056s) [seg_pos={0 0}, upload=54498]: Begin upload segment... Jul 26 20:28:22.481500645 (56h53m5.32630217s) [seg_pos={0 0}, upload=54498]: Done begin upload segment. Jul 26 20:28:22.481533046 (56h53m5.326334571s) [noise=true , address=45.91.116.4:4938 , piece_id=XPEH2CKEBJUEDAZGL4W6SZYHFCOSVP6IU2OWIU44EVU46QB7TQOA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.481538519 (56h53m5.326340049s) [noise=true , address=45.91.116.8:4047 , piece_id=QCPM7CFY5UBXWAZEAFO57NELB5R62GZ7CG2ISD2TSWAMNG6LYTRQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.481625224 (56h53m5.326426746s) [noise=true , address=77.90.31.12:4589 , piece_id=VUCKZDOASEF646TUXJEEUR2IEMVPAE4LV4I57AB6HMNQG2CZCTWA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.482010694 (56h53m5.326812226s) [noise=true , address=104.36.82.8:4969 , piece_id=EQKLKHNZE7DGZOBGZRBUA4BTVJU6AVHKHK7CZAXCEKTKULNG2E5A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.482180452 (56h53m5.326981984s) [noise=true , address=77.90.31.21:4878 , piece_id=3L4TERTQQBVRHMR2IXFGYGPVDJQK6KCPS6DFOJCI2KAVG5KBLJ3A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.482706547 (56h53m5.327508072s) [noise=true , address=77.90.31.15:4698 , piece_id=UZBDOABD2WFFPYJCC5EYJUZX6NU4REM5MRZF7POAQR3TXTZ5HXPQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.483145730 (56h53m5.327947256s) [noise=true , address=77.90.31.14:4662 , piece_id=LFCOMWII6WAEYRC3J4NMTC2RNOS3MX4WNLDUFPI4TGPWO64AFOXQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.483696003 (56h53m5.328497531s) [noise=true , address=45.91.116.12:4226 , piece_id=FY47OOHAOWN4DZCL446MZKMNEZVUFH37DEFUGOKHDAYNG3QFFRNQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.483766294 (56h53m5.328567822s) [noise=true , address=77.90.31.19:4843 , piece_id=GY3TBUVRZWDCJKIBM3GBPA2MMDPKB3VYF6CIRTWJ32MHXXNIDX7Q, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.484435977 (56h53m5.329237499s) [noise=true , address=45.91.116.13:4154 , piece_id=H6XMHU4LQO7T7NVPKAOBU3TJMJSGDS2SDZIE2KYYCXNT2MK4MFOQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.484911788 (56h53m5.329713311s) [noise=true , address=45.91.116.10:4082 , piece_id=C6G2APYQRU3LLXOZFU2PWQRXR3OQ2WYIADMJ76J2WVDY7HKNDTNA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.487491433 (56h53m5.332292958s) [noise=true , address=77.90.31.13:4626 , piece_id=KTVRUR4P2Z3N5CB26BLEIBORDOSZQTZM4D3E2IHEGZ4QXXWDBPXA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.487914119 (56h53m5.332715646s) [noise=true , address=77.90.31.21:4877 , piece_id=UHPHX5KJKOB3DGWIWTMPN7P73V76GAVCOLOO4GTPMQ6BDGTCMA4A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.488256189 (56h53m5.333057714s) [noise=true , address=45.91.116.14:4191 , piece_id=M6ZKO6X2XJ7UMFZWSGNWUIEYDTXJB6C62ZXE7SVRIF6XOMCCQD4Q, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.488930410 (56h53m5.333731944s) [noise=true , address=45.91.116.6:4974 , piece_id=3W43RVQMPBJXQJWYUYX4IJ7CZUP2SWCMXZEB4AL6ITZ3OVPMZA2A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.489313463 (56h53m5.334114987s) [noise=true , address=45.91.116.11:4119 , piece_id=DRIZSI6XSABNTXWUMNFY3RPV3HPUBVE62HXN7XG55FM44TMVCYSQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:23.003858627 (56h53m5.848660156s) [upload=54498]: Next returned nil segment Jul 26 20:28:23.003881158 (56h53m5.848682712s) [upload=54498]: Waiting for error group managing segments... Jul 26 20:28:23.117181340 (56h53m5.961982857s) [noise=true , address=77.90.31.19:4843 , piece_id=GY3TBUVRZWDCJKIBM3GBPA2MMDPKB3VYF6CIRTWJ32MHXXNIDX7Q, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.121502780 (56h53m5.966304296s) [noise=true , address=77.90.31.15:4698 , piece_id=UZBDOABD2WFFPYJCC5EYJUZX6NU4REM5MRZF7POAQR3TXTZ5HXPQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.128715466 (56h53m5.973516987s) [noise=true , address=77.90.31.12:4589 , piece_id=VUCKZDOASEF646TUXJEEUR2IEMVPAE4LV4I57AB6HMNQG2CZCTWA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.136428815 (56h53m5.981230348s) [noise=true , address=77.90.31.14:4662 , piece_id=LFCOMWII6WAEYRC3J4NMTC2RNOS3MX4WNLDUFPI4TGPWO64AFOXQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.140913043 (56h53m5.985714576s) [noise=true , address=77.90.31.13:4626 , piece_id=KTVRUR4P2Z3N5CB26BLEIBORDOSZQTZM4D3E2IHEGZ4QXXWDBPXA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.144098231 (56h53m5.988899746s) [noise=true , address=77.90.31.21:4877 , piece_id=UHPHX5KJKOB3DGWIWTMPN7P73V76GAVCOLOO4GTPMQ6BDGTCMA4A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.154118120 (56h53m5.998919628s) [noise=true , address=77.90.31.21:4878 , piece_id=3L4TERTQQBVRHMR2IXFGYGPVDJQK6KCPS6DFOJCI2KAVG5KBLJ3A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.851371196 (56h53m6.696172711s) [noise=true , address=104.36.82.8:4969 , piece_id=EQKLKHNZE7DGZOBGZRBUA4BTVJU6AVHKHK7CZAXCEKTKULNG2E5A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:52.417946210 (56h53m35.262747746s) [noise=true , address=45.91.116.8:4047 , piece_id=QCPM7CFY5UBXWAZEAFO57NELB5R62GZ7CG2ISD2TSWAMNG6LYTRQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: rpc: dial tcp 45.91.116.8:4047: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418043876 (56h53m35.26284539s) [noise=true , address=45.91.116.14:4191 , piece_id=M6ZKO6X2XJ7UMFZWSGNWUIEYDTXJB6C62ZXE7SVRIF6XOMCCQD4Q, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418090229 (56h53m35.262891746s) [noise=true , address=45.91.116.13:4154 , piece_id=H6XMHU4LQO7T7NVPKAOBU3TJMJSGDS2SDZIE2KYYCXNT2MK4MFOQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 159.223.143.39:42688->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418225695 (56h53m35.263027225s) [noise=true , address=45.91.116.10:4082 , piece_id=C6G2APYQRU3LLXOZFU2PWQRXR3OQ2WYIADMJ76J2WVDY7HKNDTNA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: write tcp 159.223.143.39:60366->45.91.116.10:4082: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418340242 (56h53m35.263141768s) [noise=true , address=45.91.116.12:4226 , piece_id=FY47OOHAOWN4DZCL446MZKMNEZVUFH37DEFUGOKHDAYNG3QFFRNQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: write tcp 159.223.143.39:37720->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418429162 (56h53m35.263230689s) [noise=true , address=45.91.116.4:4938 , piece_id=XPEH2CKEBJUEDAZGL4W6SZYHFCOSVP6IU2OWIU44EVU46QB7TQOA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418473168 (56h53m35.263274683s) [noise=true , address=45.91.116.6:4974 , piece_id=3W43RVQMPBJXQJWYUYX4IJ7CZUP2SWCMXZEB4AL6ITZ3OVPMZA2A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 159.223.143.39:37724->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418564079 (56h53m35.263365602s) [noise=true , address=45.91.116.11:4119 , piece_id=DRIZSI6XSABNTXWUMNFY3RPV3HPUBVE62HXN7XG55FM44TMVCYSQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:44972->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418628408 (56h53m35.263429934s) [seg_pos={0 0}, upload=54498]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:52.444538177 (56h53m35.289339693s) [upload=54498]: Done uploading segments... Jul 26 20:28:52.444571403 (56h53m35.289372917s) [upload=54498]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 527.074018ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.XX.........................................................] duration: 554.163285ms FAIL (2025-07-26 21:12:55 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== :22.446786005 (56h53m5.29158752s) [upload=54498]: Done starting upload Jul 26 20:28:22.446976619 (56h53m5.291778131s) [upload=54498]: Uploading segments... Jul 26 20:28:22.447194509 (56h53m5.291996032s) [upload=54498]: Got next segment. Inline: false Jul 26 20:28:22.447258557 (56h53m5.292060069s) [upload=54498]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:22.447264559 (56h53m5.292066138s) [upload=54498]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:22.481429047 (56h53m5.32623056s) [seg_pos={0 0}, upload=54498]: Begin upload segment... Jul 26 20:28:22.481500645 (56h53m5.32630217s) [seg_pos={0 0}, upload=54498]: Done begin upload segment. Jul 26 20:28:22.481533046 (56h53m5.326334571s) [noise=true , address=45.91.116.4:4938 , piece_id=XPEH2CKEBJUEDAZGL4W6SZYHFCOSVP6IU2OWIU44EVU46QB7TQOA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.481538519 (56h53m5.326340049s) [noise=true , address=45.91.116.8:4047 , piece_id=QCPM7CFY5UBXWAZEAFO57NELB5R62GZ7CG2ISD2TSWAMNG6LYTRQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.481625224 (56h53m5.326426746s) [noise=true , address=77.90.31.12:4589 , piece_id=VUCKZDOASEF646TUXJEEUR2IEMVPAE4LV4I57AB6HMNQG2CZCTWA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.482010694 (56h53m5.326812226s) [noise=true , address=104.36.82.8:4969 , piece_id=EQKLKHNZE7DGZOBGZRBUA4BTVJU6AVHKHK7CZAXCEKTKULNG2E5A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.482180452 (56h53m5.326981984s) [noise=true , address=77.90.31.21:4878 , piece_id=3L4TERTQQBVRHMR2IXFGYGPVDJQK6KCPS6DFOJCI2KAVG5KBLJ3A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.482706547 (56h53m5.327508072s) [noise=true , address=77.90.31.15:4698 , piece_id=UZBDOABD2WFFPYJCC5EYJUZX6NU4REM5MRZF7POAQR3TXTZ5HXPQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.483145730 (56h53m5.327947256s) [noise=true , address=77.90.31.14:4662 , piece_id=LFCOMWII6WAEYRC3J4NMTC2RNOS3MX4WNLDUFPI4TGPWO64AFOXQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.483696003 (56h53m5.328497531s) [noise=true , address=45.91.116.12:4226 , piece_id=FY47OOHAOWN4DZCL446MZKMNEZVUFH37DEFUGOKHDAYNG3QFFRNQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.483766294 (56h53m5.328567822s) [noise=true , address=77.90.31.19:4843 , piece_id=GY3TBUVRZWDCJKIBM3GBPA2MMDPKB3VYF6CIRTWJ32MHXXNIDX7Q, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.484435977 (56h53m5.329237499s) [noise=true , address=45.91.116.13:4154 , piece_id=H6XMHU4LQO7T7NVPKAOBU3TJMJSGDS2SDZIE2KYYCXNT2MK4MFOQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.484911788 (56h53m5.329713311s) [noise=true , address=45.91.116.10:4082 , piece_id=C6G2APYQRU3LLXOZFU2PWQRXR3OQ2WYIADMJ76J2WVDY7HKNDTNA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.487491433 (56h53m5.332292958s) [noise=true , address=77.90.31.13:4626 , piece_id=KTVRUR4P2Z3N5CB26BLEIBORDOSZQTZM4D3E2IHEGZ4QXXWDBPXA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.487914119 (56h53m5.332715646s) [noise=true , address=77.90.31.21:4877 , piece_id=UHPHX5KJKOB3DGWIWTMPN7P73V76GAVCOLOO4GTPMQ6BDGTCMA4A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.488256189 (56h53m5.333057714s) [noise=true , address=45.91.116.14:4191 , piece_id=M6ZKO6X2XJ7UMFZWSGNWUIEYDTXJB6C62ZXE7SVRIF6XOMCCQD4Q, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.488930410 (56h53m5.333731944s) [noise=true , address=45.91.116.6:4974 , piece_id=3W43RVQMPBJXQJWYUYX4IJ7CZUP2SWCMXZEB4AL6ITZ3OVPMZA2A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.489313463 (56h53m5.334114987s) [noise=true , address=45.91.116.11:4119 , piece_id=DRIZSI6XSABNTXWUMNFY3RPV3HPUBVE62HXN7XG55FM44TMVCYSQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:23.003858627 (56h53m5.848660156s) [upload=54498]: Next returned nil segment Jul 26 20:28:23.003881158 (56h53m5.848682712s) [upload=54498]: Waiting for error group managing segments... Jul 26 20:28:23.117181340 (56h53m5.961982857s) [noise=true , address=77.90.31.19:4843 , piece_id=GY3TBUVRZWDCJKIBM3GBPA2MMDPKB3VYF6CIRTWJ32MHXXNIDX7Q, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.121502780 (56h53m5.966304296s) [noise=true , address=77.90.31.15:4698 , piece_id=UZBDOABD2WFFPYJCC5EYJUZX6NU4REM5MRZF7POAQR3TXTZ5HXPQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.128715466 (56h53m5.973516987s) [noise=true , address=77.90.31.12:4589 , piece_id=VUCKZDOASEF646TUXJEEUR2IEMVPAE4LV4I57AB6HMNQG2CZCTWA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.136428815 (56h53m5.981230348s) [noise=true , address=77.90.31.14:4662 , piece_id=LFCOMWII6WAEYRC3J4NMTC2RNOS3MX4WNLDUFPI4TGPWO64AFOXQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.140913043 (56h53m5.985714576s) [noise=true , address=77.90.31.13:4626 , piece_id=KTVRUR4P2Z3N5CB26BLEIBORDOSZQTZM4D3E2IHEGZ4QXXWDBPXA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.144098231 (56h53m5.988899746s) [noise=true , address=77.90.31.21:4877 , piece_id=UHPHX5KJKOB3DGWIWTMPN7P73V76GAVCOLOO4GTPMQ6BDGTCMA4A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.154118120 (56h53m5.998919628s) [noise=true , address=77.90.31.21:4878 , piece_id=3L4TERTQQBVRHMR2IXFGYGPVDJQK6KCPS6DFOJCI2KAVG5KBLJ3A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.851371196 (56h53m6.696172711s) [noise=true , address=104.36.82.8:4969 , piece_id=EQKLKHNZE7DGZOBGZRBUA4BTVJU6AVHKHK7CZAXCEKTKULNG2E5A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:52.417946210 (56h53m35.262747746s) [noise=true , address=45.91.116.8:4047 , piece_id=QCPM7CFY5UBXWAZEAFO57NELB5R62GZ7CG2ISD2TSWAMNG6LYTRQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: rpc: dial tcp 45.91.116.8:4047: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418043876 (56h53m35.26284539s) [noise=true , address=45.91.116.14:4191 , piece_id=M6ZKO6X2XJ7UMFZWSGNWUIEYDTXJB6C62ZXE7SVRIF6XOMCCQD4Q, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418090229 (56h53m35.262891746s) [noise=true , address=45.91.116.13:4154 , piece_id=H6XMHU4LQO7T7NVPKAOBU3TJMJSGDS2SDZIE2KYYCXNT2MK4MFOQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 159.223.143.39:42688->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418225695 (56h53m35.263027225s) [noise=true , address=45.91.116.10:4082 , piece_id=C6G2APYQRU3LLXOZFU2PWQRXR3OQ2WYIADMJ76J2WVDY7HKNDTNA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: write tcp 159.223.143.39:60366->45.91.116.10:4082: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418340242 (56h53m35.263141768s) [noise=true , address=45.91.116.12:4226 , piece_id=FY47OOHAOWN4DZCL446MZKMNEZVUFH37DEFUGOKHDAYNG3QFFRNQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: write tcp 159.223.143.39:37720->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418429162 (56h53m35.263230689s) [noise=true , address=45.91.116.4:4938 , piece_id=XPEH2CKEBJUEDAZGL4W6SZYHFCOSVP6IU2OWIU44EVU46QB7TQOA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418473168 (56h53m35.263274683s) [noise=true , address=45.91.116.6:4974 , piece_id=3W43RVQMPBJXQJWYUYX4IJ7CZUP2SWCMXZEB4AL6ITZ3OVPMZA2A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 159.223.143.39:37724->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418564079 (56h53m35.263365602s) [noise=true , address=45.91.116.11:4119 , piece_id=DRIZSI6XSABNTXWUMNFY3RPV3HPUBVE62HXN7XG55FM44TMVCYSQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:44972->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418628408 (56h53m35.263429934s) [seg_pos={0 0}, upload=54498]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:52.444538177 (56h53m35.289339693s) [upload=54498]: Done uploading segments... Jul 26 20:28:52.444571403 (56h53m35.289372917s) [upload=54498]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 554.197203ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 210.274488ms FAIL (2025-07-26 21:12:13 UTC) orbiter warning: uplink: too many requests Duration: 210.302789ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..XX........................................................] duration: 683.680695ms FAIL (2025-07-26 21:11:53 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== :22.446786005 (56h53m5.29158752s) [upload=54498]: Done starting upload Jul 26 20:28:22.446976619 (56h53m5.291778131s) [upload=54498]: Uploading segments... Jul 26 20:28:22.447194509 (56h53m5.291996032s) [upload=54498]: Got next segment. Inline: false Jul 26 20:28:22.447258557 (56h53m5.292060069s) [upload=54498]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:22.447264559 (56h53m5.292066138s) [upload=54498]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:22.481429047 (56h53m5.32623056s) [seg_pos={0 0}, upload=54498]: Begin upload segment... Jul 26 20:28:22.481500645 (56h53m5.32630217s) [seg_pos={0 0}, upload=54498]: Done begin upload segment. Jul 26 20:28:22.481533046 (56h53m5.326334571s) [noise=true , address=45.91.116.4:4938 , piece_id=XPEH2CKEBJUEDAZGL4W6SZYHFCOSVP6IU2OWIU44EVU46QB7TQOA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.481538519 (56h53m5.326340049s) [noise=true , address=45.91.116.8:4047 , piece_id=QCPM7CFY5UBXWAZEAFO57NELB5R62GZ7CG2ISD2TSWAMNG6LYTRQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.481625224 (56h53m5.326426746s) [noise=true , address=77.90.31.12:4589 , piece_id=VUCKZDOASEF646TUXJEEUR2IEMVPAE4LV4I57AB6HMNQG2CZCTWA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.482010694 (56h53m5.326812226s) [noise=true , address=104.36.82.8:4969 , piece_id=EQKLKHNZE7DGZOBGZRBUA4BTVJU6AVHKHK7CZAXCEKTKULNG2E5A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.482180452 (56h53m5.326981984s) [noise=true , address=77.90.31.21:4878 , piece_id=3L4TERTQQBVRHMR2IXFGYGPVDJQK6KCPS6DFOJCI2KAVG5KBLJ3A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.482706547 (56h53m5.327508072s) [noise=true , address=77.90.31.15:4698 , piece_id=UZBDOABD2WFFPYJCC5EYJUZX6NU4REM5MRZF7POAQR3TXTZ5HXPQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.483145730 (56h53m5.327947256s) [noise=true , address=77.90.31.14:4662 , piece_id=LFCOMWII6WAEYRC3J4NMTC2RNOS3MX4WNLDUFPI4TGPWO64AFOXQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.483696003 (56h53m5.328497531s) [noise=true , address=45.91.116.12:4226 , piece_id=FY47OOHAOWN4DZCL446MZKMNEZVUFH37DEFUGOKHDAYNG3QFFRNQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.483766294 (56h53m5.328567822s) [noise=true , address=77.90.31.19:4843 , piece_id=GY3TBUVRZWDCJKIBM3GBPA2MMDPKB3VYF6CIRTWJ32MHXXNIDX7Q, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.484435977 (56h53m5.329237499s) [noise=true , address=45.91.116.13:4154 , piece_id=H6XMHU4LQO7T7NVPKAOBU3TJMJSGDS2SDZIE2KYYCXNT2MK4MFOQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.484911788 (56h53m5.329713311s) [noise=true , address=45.91.116.10:4082 , piece_id=C6G2APYQRU3LLXOZFU2PWQRXR3OQ2WYIADMJ76J2WVDY7HKNDTNA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.487491433 (56h53m5.332292958s) [noise=true , address=77.90.31.13:4626 , piece_id=KTVRUR4P2Z3N5CB26BLEIBORDOSZQTZM4D3E2IHEGZ4QXXWDBPXA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.487914119 (56h53m5.332715646s) [noise=true , address=77.90.31.21:4877 , piece_id=UHPHX5KJKOB3DGWIWTMPN7P73V76GAVCOLOO4GTPMQ6BDGTCMA4A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.488256189 (56h53m5.333057714s) [noise=true , address=45.91.116.14:4191 , piece_id=M6ZKO6X2XJ7UMFZWSGNWUIEYDTXJB6C62ZXE7SVRIF6XOMCCQD4Q, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.488930410 (56h53m5.333731944s) [noise=true , address=45.91.116.6:4974 , piece_id=3W43RVQMPBJXQJWYUYX4IJ7CZUP2SWCMXZEB4AL6ITZ3OVPMZA2A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.489313463 (56h53m5.334114987s) [noise=true , address=45.91.116.11:4119 , piece_id=DRIZSI6XSABNTXWUMNFY3RPV3HPUBVE62HXN7XG55FM44TMVCYSQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:23.003858627 (56h53m5.848660156s) [upload=54498]: Next returned nil segment Jul 26 20:28:23.003881158 (56h53m5.848682712s) [upload=54498]: Waiting for error group managing segments... Jul 26 20:28:23.117181340 (56h53m5.961982857s) [noise=true , address=77.90.31.19:4843 , piece_id=GY3TBUVRZWDCJKIBM3GBPA2MMDPKB3VYF6CIRTWJ32MHXXNIDX7Q, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.121502780 (56h53m5.966304296s) [noise=true , address=77.90.31.15:4698 , piece_id=UZBDOABD2WFFPYJCC5EYJUZX6NU4REM5MRZF7POAQR3TXTZ5HXPQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.128715466 (56h53m5.973516987s) [noise=true , address=77.90.31.12:4589 , piece_id=VUCKZDOASEF646TUXJEEUR2IEMVPAE4LV4I57AB6HMNQG2CZCTWA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.136428815 (56h53m5.981230348s) [noise=true , address=77.90.31.14:4662 , piece_id=LFCOMWII6WAEYRC3J4NMTC2RNOS3MX4WNLDUFPI4TGPWO64AFOXQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.140913043 (56h53m5.985714576s) [noise=true , address=77.90.31.13:4626 , piece_id=KTVRUR4P2Z3N5CB26BLEIBORDOSZQTZM4D3E2IHEGZ4QXXWDBPXA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.144098231 (56h53m5.988899746s) [noise=true , address=77.90.31.21:4877 , piece_id=UHPHX5KJKOB3DGWIWTMPN7P73V76GAVCOLOO4GTPMQ6BDGTCMA4A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.154118120 (56h53m5.998919628s) [noise=true , address=77.90.31.21:4878 , piece_id=3L4TERTQQBVRHMR2IXFGYGPVDJQK6KCPS6DFOJCI2KAVG5KBLJ3A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.851371196 (56h53m6.696172711s) [noise=true , address=104.36.82.8:4969 , piece_id=EQKLKHNZE7DGZOBGZRBUA4BTVJU6AVHKHK7CZAXCEKTKULNG2E5A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:52.417946210 (56h53m35.262747746s) [noise=true , address=45.91.116.8:4047 , piece_id=QCPM7CFY5UBXWAZEAFO57NELB5R62GZ7CG2ISD2TSWAMNG6LYTRQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: rpc: dial tcp 45.91.116.8:4047: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418043876 (56h53m35.26284539s) [noise=true , address=45.91.116.14:4191 , piece_id=M6ZKO6X2XJ7UMFZWSGNWUIEYDTXJB6C62ZXE7SVRIF6XOMCCQD4Q, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418090229 (56h53m35.262891746s) [noise=true , address=45.91.116.13:4154 , piece_id=H6XMHU4LQO7T7NVPKAOBU3TJMJSGDS2SDZIE2KYYCXNT2MK4MFOQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 159.223.143.39:42688->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418225695 (56h53m35.263027225s) [noise=true , address=45.91.116.10:4082 , piece_id=C6G2APYQRU3LLXOZFU2PWQRXR3OQ2WYIADMJ76J2WVDY7HKNDTNA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: write tcp 159.223.143.39:60366->45.91.116.10:4082: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418340242 (56h53m35.263141768s) [noise=true , address=45.91.116.12:4226 , piece_id=FY47OOHAOWN4DZCL446MZKMNEZVUFH37DEFUGOKHDAYNG3QFFRNQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: write tcp 159.223.143.39:37720->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418429162 (56h53m35.263230689s) [noise=true , address=45.91.116.4:4938 , piece_id=XPEH2CKEBJUEDAZGL4W6SZYHFCOSVP6IU2OWIU44EVU46QB7TQOA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418473168 (56h53m35.263274683s) [noise=true , address=45.91.116.6:4974 , piece_id=3W43RVQMPBJXQJWYUYX4IJ7CZUP2SWCMXZEB4AL6ITZ3OVPMZA2A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 159.223.143.39:37724->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418564079 (56h53m35.263365602s) [noise=true , address=45.91.116.11:4119 , piece_id=DRIZSI6XSABNTXWUMNFY3RPV3HPUBVE62HXN7XG55FM44TMVCYSQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:44972->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418628408 (56h53m35.263429934s) [seg_pos={0 0}, upload=54498]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:52.444538177 (56h53m35.289339693s) [upload=54498]: Done uploading segments... Jul 26 20:28:52.444571403 (56h53m35.289372917s) [upload=54498]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 683.717373ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 280.638468ms FAIL (2025-07-26 21:11:26 UTC) orbiter warning: uplink: too many requests Duration: 280.691264ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...XX.......................................................] duration: 513.703722ms FAIL (2025-07-26 21:11:17 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== :22.446786005 (56h53m5.29158752s) [upload=54498]: Done starting upload Jul 26 20:28:22.446976619 (56h53m5.291778131s) [upload=54498]: Uploading segments... Jul 26 20:28:22.447194509 (56h53m5.291996032s) [upload=54498]: Got next segment. Inline: false Jul 26 20:28:22.447258557 (56h53m5.292060069s) [upload=54498]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:22.447264559 (56h53m5.292066138s) [upload=54498]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:22.481429047 (56h53m5.32623056s) [seg_pos={0 0}, upload=54498]: Begin upload segment... Jul 26 20:28:22.481500645 (56h53m5.32630217s) [seg_pos={0 0}, upload=54498]: Done begin upload segment. Jul 26 20:28:22.481533046 (56h53m5.326334571s) [noise=true , address=45.91.116.4:4938 , piece_id=XPEH2CKEBJUEDAZGL4W6SZYHFCOSVP6IU2OWIU44EVU46QB7TQOA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.481538519 (56h53m5.326340049s) [noise=true , address=45.91.116.8:4047 , piece_id=QCPM7CFY5UBXWAZEAFO57NELB5R62GZ7CG2ISD2TSWAMNG6LYTRQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.481625224 (56h53m5.326426746s) [noise=true , address=77.90.31.12:4589 , piece_id=VUCKZDOASEF646TUXJEEUR2IEMVPAE4LV4I57AB6HMNQG2CZCTWA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.482010694 (56h53m5.326812226s) [noise=true , address=104.36.82.8:4969 , piece_id=EQKLKHNZE7DGZOBGZRBUA4BTVJU6AVHKHK7CZAXCEKTKULNG2E5A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.482180452 (56h53m5.326981984s) [noise=true , address=77.90.31.21:4878 , piece_id=3L4TERTQQBVRHMR2IXFGYGPVDJQK6KCPS6DFOJCI2KAVG5KBLJ3A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.482706547 (56h53m5.327508072s) [noise=true , address=77.90.31.15:4698 , piece_id=UZBDOABD2WFFPYJCC5EYJUZX6NU4REM5MRZF7POAQR3TXTZ5HXPQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.483145730 (56h53m5.327947256s) [noise=true , address=77.90.31.14:4662 , piece_id=LFCOMWII6WAEYRC3J4NMTC2RNOS3MX4WNLDUFPI4TGPWO64AFOXQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.483696003 (56h53m5.328497531s) [noise=true , address=45.91.116.12:4226 , piece_id=FY47OOHAOWN4DZCL446MZKMNEZVUFH37DEFUGOKHDAYNG3QFFRNQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.483766294 (56h53m5.328567822s) [noise=true , address=77.90.31.19:4843 , piece_id=GY3TBUVRZWDCJKIBM3GBPA2MMDPKB3VYF6CIRTWJ32MHXXNIDX7Q, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.484435977 (56h53m5.329237499s) [noise=true , address=45.91.116.13:4154 , piece_id=H6XMHU4LQO7T7NVPKAOBU3TJMJSGDS2SDZIE2KYYCXNT2MK4MFOQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.484911788 (56h53m5.329713311s) [noise=true , address=45.91.116.10:4082 , piece_id=C6G2APYQRU3LLXOZFU2PWQRXR3OQ2WYIADMJ76J2WVDY7HKNDTNA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.487491433 (56h53m5.332292958s) [noise=true , address=77.90.31.13:4626 , piece_id=KTVRUR4P2Z3N5CB26BLEIBORDOSZQTZM4D3E2IHEGZ4QXXWDBPXA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.487914119 (56h53m5.332715646s) [noise=true , address=77.90.31.21:4877 , piece_id=UHPHX5KJKOB3DGWIWTMPN7P73V76GAVCOLOO4GTPMQ6BDGTCMA4A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.488256189 (56h53m5.333057714s) [noise=true , address=45.91.116.14:4191 , piece_id=M6ZKO6X2XJ7UMFZWSGNWUIEYDTXJB6C62ZXE7SVRIF6XOMCCQD4Q, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.488930410 (56h53m5.333731944s) [noise=true , address=45.91.116.6:4974 , piece_id=3W43RVQMPBJXQJWYUYX4IJ7CZUP2SWCMXZEB4AL6ITZ3OVPMZA2A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.489313463 (56h53m5.334114987s) [noise=true , address=45.91.116.11:4119 , piece_id=DRIZSI6XSABNTXWUMNFY3RPV3HPUBVE62HXN7XG55FM44TMVCYSQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:23.003858627 (56h53m5.848660156s) [upload=54498]: Next returned nil segment Jul 26 20:28:23.003881158 (56h53m5.848682712s) [upload=54498]: Waiting for error group managing segments... Jul 26 20:28:23.117181340 (56h53m5.961982857s) [noise=true , address=77.90.31.19:4843 , piece_id=GY3TBUVRZWDCJKIBM3GBPA2MMDPKB3VYF6CIRTWJ32MHXXNIDX7Q, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.121502780 (56h53m5.966304296s) [noise=true , address=77.90.31.15:4698 , piece_id=UZBDOABD2WFFPYJCC5EYJUZX6NU4REM5MRZF7POAQR3TXTZ5HXPQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.128715466 (56h53m5.973516987s) [noise=true , address=77.90.31.12:4589 , piece_id=VUCKZDOASEF646TUXJEEUR2IEMVPAE4LV4I57AB6HMNQG2CZCTWA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.136428815 (56h53m5.981230348s) [noise=true , address=77.90.31.14:4662 , piece_id=LFCOMWII6WAEYRC3J4NMTC2RNOS3MX4WNLDUFPI4TGPWO64AFOXQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.140913043 (56h53m5.985714576s) [noise=true , address=77.90.31.13:4626 , piece_id=KTVRUR4P2Z3N5CB26BLEIBORDOSZQTZM4D3E2IHEGZ4QXXWDBPXA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.144098231 (56h53m5.988899746s) [noise=true , address=77.90.31.21:4877 , piece_id=UHPHX5KJKOB3DGWIWTMPN7P73V76GAVCOLOO4GTPMQ6BDGTCMA4A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.154118120 (56h53m5.998919628s) [noise=true , address=77.90.31.21:4878 , piece_id=3L4TERTQQBVRHMR2IXFGYGPVDJQK6KCPS6DFOJCI2KAVG5KBLJ3A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.851371196 (56h53m6.696172711s) [noise=true , address=104.36.82.8:4969 , piece_id=EQKLKHNZE7DGZOBGZRBUA4BTVJU6AVHKHK7CZAXCEKTKULNG2E5A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:52.417946210 (56h53m35.262747746s) [noise=true , address=45.91.116.8:4047 , piece_id=QCPM7CFY5UBXWAZEAFO57NELB5R62GZ7CG2ISD2TSWAMNG6LYTRQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: rpc: dial tcp 45.91.116.8:4047: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418043876 (56h53m35.26284539s) [noise=true , address=45.91.116.14:4191 , piece_id=M6ZKO6X2XJ7UMFZWSGNWUIEYDTXJB6C62ZXE7SVRIF6XOMCCQD4Q, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418090229 (56h53m35.262891746s) [noise=true , address=45.91.116.13:4154 , piece_id=H6XMHU4LQO7T7NVPKAOBU3TJMJSGDS2SDZIE2KYYCXNT2MK4MFOQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 159.223.143.39:42688->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418225695 (56h53m35.263027225s) [noise=true , address=45.91.116.10:4082 , piece_id=C6G2APYQRU3LLXOZFU2PWQRXR3OQ2WYIADMJ76J2WVDY7HKNDTNA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: write tcp 159.223.143.39:60366->45.91.116.10:4082: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418340242 (56h53m35.263141768s) [noise=true , address=45.91.116.12:4226 , piece_id=FY47OOHAOWN4DZCL446MZKMNEZVUFH37DEFUGOKHDAYNG3QFFRNQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: write tcp 159.223.143.39:37720->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418429162 (56h53m35.263230689s) [noise=true , address=45.91.116.4:4938 , piece_id=XPEH2CKEBJUEDAZGL4W6SZYHFCOSVP6IU2OWIU44EVU46QB7TQOA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418473168 (56h53m35.263274683s) [noise=true , address=45.91.116.6:4974 , piece_id=3W43RVQMPBJXQJWYUYX4IJ7CZUP2SWCMXZEB4AL6ITZ3OVPMZA2A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 159.223.143.39:37724->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418564079 (56h53m35.263365602s) [noise=true , address=45.91.116.11:4119 , piece_id=DRIZSI6XSABNTXWUMNFY3RPV3HPUBVE62HXN7XG55FM44TMVCYSQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:44972->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418628408 (56h53m35.263429934s) [seg_pos={0 0}, upload=54498]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:52.444538177 (56h53m35.289339693s) [upload=54498]: Done uploading segments... Jul 26 20:28:52.444571403 (56h53m35.289372917s) [upload=54498]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 513.735087ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 373.064259ms FAIL (2025-07-26 21:11:06 UTC) orbiter warning: uplink: too many requests Duration: 373.097139ms core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 1.61828929s FAIL (2025-07-26 21:10:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.618322391s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....XX......................................................] duration: 1.137060335s FAIL (2025-07-26 21:10:31 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== :22.446786005 (56h53m5.29158752s) [upload=54498]: Done starting upload Jul 26 20:28:22.446976619 (56h53m5.291778131s) [upload=54498]: Uploading segments... Jul 26 20:28:22.447194509 (56h53m5.291996032s) [upload=54498]: Got next segment. Inline: false Jul 26 20:28:22.447258557 (56h53m5.292060069s) [upload=54498]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:22.447264559 (56h53m5.292066138s) [upload=54498]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:22.481429047 (56h53m5.32623056s) [seg_pos={0 0}, upload=54498]: Begin upload segment... Jul 26 20:28:22.481500645 (56h53m5.32630217s) [seg_pos={0 0}, upload=54498]: Done begin upload segment. Jul 26 20:28:22.481533046 (56h53m5.326334571s) [noise=true , address=45.91.116.4:4938 , piece_id=XPEH2CKEBJUEDAZGL4W6SZYHFCOSVP6IU2OWIU44EVU46QB7TQOA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.481538519 (56h53m5.326340049s) [noise=true , address=45.91.116.8:4047 , piece_id=QCPM7CFY5UBXWAZEAFO57NELB5R62GZ7CG2ISD2TSWAMNG6LYTRQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.481625224 (56h53m5.326426746s) [noise=true , address=77.90.31.12:4589 , piece_id=VUCKZDOASEF646TUXJEEUR2IEMVPAE4LV4I57AB6HMNQG2CZCTWA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.482010694 (56h53m5.326812226s) [noise=true , address=104.36.82.8:4969 , piece_id=EQKLKHNZE7DGZOBGZRBUA4BTVJU6AVHKHK7CZAXCEKTKULNG2E5A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.482180452 (56h53m5.326981984s) [noise=true , address=77.90.31.21:4878 , piece_id=3L4TERTQQBVRHMR2IXFGYGPVDJQK6KCPS6DFOJCI2KAVG5KBLJ3A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.482706547 (56h53m5.327508072s) [noise=true , address=77.90.31.15:4698 , piece_id=UZBDOABD2WFFPYJCC5EYJUZX6NU4REM5MRZF7POAQR3TXTZ5HXPQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.483145730 (56h53m5.327947256s) [noise=true , address=77.90.31.14:4662 , piece_id=LFCOMWII6WAEYRC3J4NMTC2RNOS3MX4WNLDUFPI4TGPWO64AFOXQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.483696003 (56h53m5.328497531s) [noise=true , address=45.91.116.12:4226 , piece_id=FY47OOHAOWN4DZCL446MZKMNEZVUFH37DEFUGOKHDAYNG3QFFRNQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.483766294 (56h53m5.328567822s) [noise=true , address=77.90.31.19:4843 , piece_id=GY3TBUVRZWDCJKIBM3GBPA2MMDPKB3VYF6CIRTWJ32MHXXNIDX7Q, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.484435977 (56h53m5.329237499s) [noise=true , address=45.91.116.13:4154 , piece_id=H6XMHU4LQO7T7NVPKAOBU3TJMJSGDS2SDZIE2KYYCXNT2MK4MFOQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.484911788 (56h53m5.329713311s) [noise=true , address=45.91.116.10:4082 , piece_id=C6G2APYQRU3LLXOZFU2PWQRXR3OQ2WYIADMJ76J2WVDY7HKNDTNA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.487491433 (56h53m5.332292958s) [noise=true , address=77.90.31.13:4626 , piece_id=KTVRUR4P2Z3N5CB26BLEIBORDOSZQTZM4D3E2IHEGZ4QXXWDBPXA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.487914119 (56h53m5.332715646s) [noise=true , address=77.90.31.21:4877 , piece_id=UHPHX5KJKOB3DGWIWTMPN7P73V76GAVCOLOO4GTPMQ6BDGTCMA4A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.488256189 (56h53m5.333057714s) [noise=true , address=45.91.116.14:4191 , piece_id=M6ZKO6X2XJ7UMFZWSGNWUIEYDTXJB6C62ZXE7SVRIF6XOMCCQD4Q, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.488930410 (56h53m5.333731944s) [noise=true , address=45.91.116.6:4974 , piece_id=3W43RVQMPBJXQJWYUYX4IJ7CZUP2SWCMXZEB4AL6ITZ3OVPMZA2A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.489313463 (56h53m5.334114987s) [noise=true , address=45.91.116.11:4119 , piece_id=DRIZSI6XSABNTXWUMNFY3RPV3HPUBVE62HXN7XG55FM44TMVCYSQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:23.003858627 (56h53m5.848660156s) [upload=54498]: Next returned nil segment Jul 26 20:28:23.003881158 (56h53m5.848682712s) [upload=54498]: Waiting for error group managing segments... Jul 26 20:28:23.117181340 (56h53m5.961982857s) [noise=true , address=77.90.31.19:4843 , piece_id=GY3TBUVRZWDCJKIBM3GBPA2MMDPKB3VYF6CIRTWJ32MHXXNIDX7Q, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.121502780 (56h53m5.966304296s) [noise=true , address=77.90.31.15:4698 , piece_id=UZBDOABD2WFFPYJCC5EYJUZX6NU4REM5MRZF7POAQR3TXTZ5HXPQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.128715466 (56h53m5.973516987s) [noise=true , address=77.90.31.12:4589 , piece_id=VUCKZDOASEF646TUXJEEUR2IEMVPAE4LV4I57AB6HMNQG2CZCTWA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.136428815 (56h53m5.981230348s) [noise=true , address=77.90.31.14:4662 , piece_id=LFCOMWII6WAEYRC3J4NMTC2RNOS3MX4WNLDUFPI4TGPWO64AFOXQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.140913043 (56h53m5.985714576s) [noise=true , address=77.90.31.13:4626 , piece_id=KTVRUR4P2Z3N5CB26BLEIBORDOSZQTZM4D3E2IHEGZ4QXXWDBPXA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.144098231 (56h53m5.988899746s) [noise=true , address=77.90.31.21:4877 , piece_id=UHPHX5KJKOB3DGWIWTMPN7P73V76GAVCOLOO4GTPMQ6BDGTCMA4A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.154118120 (56h53m5.998919628s) [noise=true , address=77.90.31.21:4878 , piece_id=3L4TERTQQBVRHMR2IXFGYGPVDJQK6KCPS6DFOJCI2KAVG5KBLJ3A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.851371196 (56h53m6.696172711s) [noise=true , address=104.36.82.8:4969 , piece_id=EQKLKHNZE7DGZOBGZRBUA4BTVJU6AVHKHK7CZAXCEKTKULNG2E5A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:52.417946210 (56h53m35.262747746s) [noise=true , address=45.91.116.8:4047 , piece_id=QCPM7CFY5UBXWAZEAFO57NELB5R62GZ7CG2ISD2TSWAMNG6LYTRQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: rpc: dial tcp 45.91.116.8:4047: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418043876 (56h53m35.26284539s) [noise=true , address=45.91.116.14:4191 , piece_id=M6ZKO6X2XJ7UMFZWSGNWUIEYDTXJB6C62ZXE7SVRIF6XOMCCQD4Q, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418090229 (56h53m35.262891746s) [noise=true , address=45.91.116.13:4154 , piece_id=H6XMHU4LQO7T7NVPKAOBU3TJMJSGDS2SDZIE2KYYCXNT2MK4MFOQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 159.223.143.39:42688->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418225695 (56h53m35.263027225s) [noise=true , address=45.91.116.10:4082 , piece_id=C6G2APYQRU3LLXOZFU2PWQRXR3OQ2WYIADMJ76J2WVDY7HKNDTNA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: write tcp 159.223.143.39:60366->45.91.116.10:4082: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418340242 (56h53m35.263141768s) [noise=true , address=45.91.116.12:4226 , piece_id=FY47OOHAOWN4DZCL446MZKMNEZVUFH37DEFUGOKHDAYNG3QFFRNQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: write tcp 159.223.143.39:37720->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418429162 (56h53m35.263230689s) [noise=true , address=45.91.116.4:4938 , piece_id=XPEH2CKEBJUEDAZGL4W6SZYHFCOSVP6IU2OWIU44EVU46QB7TQOA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418473168 (56h53m35.263274683s) [noise=true , address=45.91.116.6:4974 , piece_id=3W43RVQMPBJXQJWYUYX4IJ7CZUP2SWCMXZEB4AL6ITZ3OVPMZA2A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 159.223.143.39:37724->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418564079 (56h53m35.263365602s) [noise=true , address=45.91.116.11:4119 , piece_id=DRIZSI6XSABNTXWUMNFY3RPV3HPUBVE62HXN7XG55FM44TMVCYSQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:44972->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418628408 (56h53m35.263429934s) [seg_pos={0 0}, upload=54498]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:52.444538177 (56h53m35.289339693s) [upload=54498]: Done uploading segments... Jul 26 20:28:52.444571403 (56h53m35.289372917s) [upload=54498]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 1.137194742s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 239.36675ms FAIL (2025-07-26 21:10:18 UTC) orbiter warning: uplink: too many requests Duration: 239.47141ms core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 671.326928ms FAIL (2025-07-26 21:10:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 671.354645ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 229.97043ms FAIL (2025-07-26 21:09:56 UTC) orbiter warning: uplink: too many requests Duration: 229.999024ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 677.287199ms FAIL (2025-07-26 21:09:46 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 8) Duration: 677.321893ms core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.XX.........................................................] duration: 742.509947ms FAIL (2025-07-26 21:09:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 742.543194ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....XX.....................................................] duration: 503.103492ms FAIL (2025-07-26 21:09:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== :22.446786005 (56h53m5.29158752s) [upload=54498]: Done starting upload Jul 26 20:28:22.446976619 (56h53m5.291778131s) [upload=54498]: Uploading segments... Jul 26 20:28:22.447194509 (56h53m5.291996032s) [upload=54498]: Got next segment. Inline: false Jul 26 20:28:22.447258557 (56h53m5.292060069s) [upload=54498]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:22.447264559 (56h53m5.292066138s) [upload=54498]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:22.481429047 (56h53m5.32623056s) [seg_pos={0 0}, upload=54498]: Begin upload segment... Jul 26 20:28:22.481500645 (56h53m5.32630217s) [seg_pos={0 0}, upload=54498]: Done begin upload segment. Jul 26 20:28:22.481533046 (56h53m5.326334571s) [noise=true , address=45.91.116.4:4938 , piece_id=XPEH2CKEBJUEDAZGL4W6SZYHFCOSVP6IU2OWIU44EVU46QB7TQOA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.481538519 (56h53m5.326340049s) [noise=true , address=45.91.116.8:4047 , piece_id=QCPM7CFY5UBXWAZEAFO57NELB5R62GZ7CG2ISD2TSWAMNG6LYTRQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.481625224 (56h53m5.326426746s) [noise=true , address=77.90.31.12:4589 , piece_id=VUCKZDOASEF646TUXJEEUR2IEMVPAE4LV4I57AB6HMNQG2CZCTWA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.482010694 (56h53m5.326812226s) [noise=true , address=104.36.82.8:4969 , piece_id=EQKLKHNZE7DGZOBGZRBUA4BTVJU6AVHKHK7CZAXCEKTKULNG2E5A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.482180452 (56h53m5.326981984s) [noise=true , address=77.90.31.21:4878 , piece_id=3L4TERTQQBVRHMR2IXFGYGPVDJQK6KCPS6DFOJCI2KAVG5KBLJ3A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.482706547 (56h53m5.327508072s) [noise=true , address=77.90.31.15:4698 , piece_id=UZBDOABD2WFFPYJCC5EYJUZX6NU4REM5MRZF7POAQR3TXTZ5HXPQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.483145730 (56h53m5.327947256s) [noise=true , address=77.90.31.14:4662 , piece_id=LFCOMWII6WAEYRC3J4NMTC2RNOS3MX4WNLDUFPI4TGPWO64AFOXQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.483696003 (56h53m5.328497531s) [noise=true , address=45.91.116.12:4226 , piece_id=FY47OOHAOWN4DZCL446MZKMNEZVUFH37DEFUGOKHDAYNG3QFFRNQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.483766294 (56h53m5.328567822s) [noise=true , address=77.90.31.19:4843 , piece_id=GY3TBUVRZWDCJKIBM3GBPA2MMDPKB3VYF6CIRTWJ32MHXXNIDX7Q, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.484435977 (56h53m5.329237499s) [noise=true , address=45.91.116.13:4154 , piece_id=H6XMHU4LQO7T7NVPKAOBU3TJMJSGDS2SDZIE2KYYCXNT2MK4MFOQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.484911788 (56h53m5.329713311s) [noise=true , address=45.91.116.10:4082 , piece_id=C6G2APYQRU3LLXOZFU2PWQRXR3OQ2WYIADMJ76J2WVDY7HKNDTNA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.487491433 (56h53m5.332292958s) [noise=true , address=77.90.31.13:4626 , piece_id=KTVRUR4P2Z3N5CB26BLEIBORDOSZQTZM4D3E2IHEGZ4QXXWDBPXA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.487914119 (56h53m5.332715646s) [noise=true , address=77.90.31.21:4877 , piece_id=UHPHX5KJKOB3DGWIWTMPN7P73V76GAVCOLOO4GTPMQ6BDGTCMA4A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.488256189 (56h53m5.333057714s) [noise=true , address=45.91.116.14:4191 , piece_id=M6ZKO6X2XJ7UMFZWSGNWUIEYDTXJB6C62ZXE7SVRIF6XOMCCQD4Q, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.488930410 (56h53m5.333731944s) [noise=true , address=45.91.116.6:4974 , piece_id=3W43RVQMPBJXQJWYUYX4IJ7CZUP2SWCMXZEB4AL6ITZ3OVPMZA2A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.489313463 (56h53m5.334114987s) [noise=true , address=45.91.116.11:4119 , piece_id=DRIZSI6XSABNTXWUMNFY3RPV3HPUBVE62HXN7XG55FM44TMVCYSQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:23.003858627 (56h53m5.848660156s) [upload=54498]: Next returned nil segment Jul 26 20:28:23.003881158 (56h53m5.848682712s) [upload=54498]: Waiting for error group managing segments... Jul 26 20:28:23.117181340 (56h53m5.961982857s) [noise=true , address=77.90.31.19:4843 , piece_id=GY3TBUVRZWDCJKIBM3GBPA2MMDPKB3VYF6CIRTWJ32MHXXNIDX7Q, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.121502780 (56h53m5.966304296s) [noise=true , address=77.90.31.15:4698 , piece_id=UZBDOABD2WFFPYJCC5EYJUZX6NU4REM5MRZF7POAQR3TXTZ5HXPQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.128715466 (56h53m5.973516987s) [noise=true , address=77.90.31.12:4589 , piece_id=VUCKZDOASEF646TUXJEEUR2IEMVPAE4LV4I57AB6HMNQG2CZCTWA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.136428815 (56h53m5.981230348s) [noise=true , address=77.90.31.14:4662 , piece_id=LFCOMWII6WAEYRC3J4NMTC2RNOS3MX4WNLDUFPI4TGPWO64AFOXQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.140913043 (56h53m5.985714576s) [noise=true , address=77.90.31.13:4626 , piece_id=KTVRUR4P2Z3N5CB26BLEIBORDOSZQTZM4D3E2IHEGZ4QXXWDBPXA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.144098231 (56h53m5.988899746s) [noise=true , address=77.90.31.21:4877 , piece_id=UHPHX5KJKOB3DGWIWTMPN7P73V76GAVCOLOO4GTPMQ6BDGTCMA4A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.154118120 (56h53m5.998919628s) [noise=true , address=77.90.31.21:4878 , piece_id=3L4TERTQQBVRHMR2IXFGYGPVDJQK6KCPS6DFOJCI2KAVG5KBLJ3A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.851371196 (56h53m6.696172711s) [noise=true , address=104.36.82.8:4969 , piece_id=EQKLKHNZE7DGZOBGZRBUA4BTVJU6AVHKHK7CZAXCEKTKULNG2E5A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:52.417946210 (56h53m35.262747746s) [noise=true , address=45.91.116.8:4047 , piece_id=QCPM7CFY5UBXWAZEAFO57NELB5R62GZ7CG2ISD2TSWAMNG6LYTRQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: rpc: dial tcp 45.91.116.8:4047: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418043876 (56h53m35.26284539s) [noise=true , address=45.91.116.14:4191 , piece_id=M6ZKO6X2XJ7UMFZWSGNWUIEYDTXJB6C62ZXE7SVRIF6XOMCCQD4Q, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418090229 (56h53m35.262891746s) [noise=true , address=45.91.116.13:4154 , piece_id=H6XMHU4LQO7T7NVPKAOBU3TJMJSGDS2SDZIE2KYYCXNT2MK4MFOQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 159.223.143.39:42688->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418225695 (56h53m35.263027225s) [noise=true , address=45.91.116.10:4082 , piece_id=C6G2APYQRU3LLXOZFU2PWQRXR3OQ2WYIADMJ76J2WVDY7HKNDTNA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: write tcp 159.223.143.39:60366->45.91.116.10:4082: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418340242 (56h53m35.263141768s) [noise=true , address=45.91.116.12:4226 , piece_id=FY47OOHAOWN4DZCL446MZKMNEZVUFH37DEFUGOKHDAYNG3QFFRNQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: write tcp 159.223.143.39:37720->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418429162 (56h53m35.263230689s) [noise=true , address=45.91.116.4:4938 , piece_id=XPEH2CKEBJUEDAZGL4W6SZYHFCOSVP6IU2OWIU44EVU46QB7TQOA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418473168 (56h53m35.263274683s) [noise=true , address=45.91.116.6:4974 , piece_id=3W43RVQMPBJXQJWYUYX4IJ7CZUP2SWCMXZEB4AL6ITZ3OVPMZA2A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 159.223.143.39:37724->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418564079 (56h53m35.263365602s) [noise=true , address=45.91.116.11:4119 , piece_id=DRIZSI6XSABNTXWUMNFY3RPV3HPUBVE62HXN7XG55FM44TMVCYSQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:44972->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418628408 (56h53m35.263429934s) [seg_pos={0 0}, upload=54498]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:52.444538177 (56h53m35.289339693s) [upload=54498]: Done uploading segments... Jul 26 20:28:52.444571403 (56h53m35.289372917s) [upload=54498]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 503.209088ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 1.722952839s FAIL (2025-07-26 21:09:22 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 8) Duration: 1.722981448s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 563.298504ms FAIL (2025-07-26 21:09:10 UTC) orbiter warning: uplink: too many requests Duration: 563.31949ms core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..XX........................................................] duration: 879.804264ms FAIL (2025-07-26 21:08:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 879.830614ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.XX.........................................................] duration: 708.865944ms FAIL (2025-07-26 21:08:44 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 8) Duration: 708.908583ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......XX....................................................] duration: 717.622742ms FAIL (2025-07-26 21:08:19 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== :22.446786005 (56h53m5.29158752s) [upload=54498]: Done starting upload Jul 26 20:28:22.446976619 (56h53m5.291778131s) [upload=54498]: Uploading segments... Jul 26 20:28:22.447194509 (56h53m5.291996032s) [upload=54498]: Got next segment. Inline: false Jul 26 20:28:22.447258557 (56h53m5.292060069s) [upload=54498]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:22.447264559 (56h53m5.292066138s) [upload=54498]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:22.481429047 (56h53m5.32623056s) [seg_pos={0 0}, upload=54498]: Begin upload segment... Jul 26 20:28:22.481500645 (56h53m5.32630217s) [seg_pos={0 0}, upload=54498]: Done begin upload segment. Jul 26 20:28:22.481533046 (56h53m5.326334571s) [noise=true , address=45.91.116.4:4938 , piece_id=XPEH2CKEBJUEDAZGL4W6SZYHFCOSVP6IU2OWIU44EVU46QB7TQOA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.481538519 (56h53m5.326340049s) [noise=true , address=45.91.116.8:4047 , piece_id=QCPM7CFY5UBXWAZEAFO57NELB5R62GZ7CG2ISD2TSWAMNG6LYTRQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.481625224 (56h53m5.326426746s) [noise=true , address=77.90.31.12:4589 , piece_id=VUCKZDOASEF646TUXJEEUR2IEMVPAE4LV4I57AB6HMNQG2CZCTWA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.482010694 (56h53m5.326812226s) [noise=true , address=104.36.82.8:4969 , piece_id=EQKLKHNZE7DGZOBGZRBUA4BTVJU6AVHKHK7CZAXCEKTKULNG2E5A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.482180452 (56h53m5.326981984s) [noise=true , address=77.90.31.21:4878 , piece_id=3L4TERTQQBVRHMR2IXFGYGPVDJQK6KCPS6DFOJCI2KAVG5KBLJ3A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.482706547 (56h53m5.327508072s) [noise=true , address=77.90.31.15:4698 , piece_id=UZBDOABD2WFFPYJCC5EYJUZX6NU4REM5MRZF7POAQR3TXTZ5HXPQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.483145730 (56h53m5.327947256s) [noise=true , address=77.90.31.14:4662 , piece_id=LFCOMWII6WAEYRC3J4NMTC2RNOS3MX4WNLDUFPI4TGPWO64AFOXQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.483696003 (56h53m5.328497531s) [noise=true , address=45.91.116.12:4226 , piece_id=FY47OOHAOWN4DZCL446MZKMNEZVUFH37DEFUGOKHDAYNG3QFFRNQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.483766294 (56h53m5.328567822s) [noise=true , address=77.90.31.19:4843 , piece_id=GY3TBUVRZWDCJKIBM3GBPA2MMDPKB3VYF6CIRTWJ32MHXXNIDX7Q, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.484435977 (56h53m5.329237499s) [noise=true , address=45.91.116.13:4154 , piece_id=H6XMHU4LQO7T7NVPKAOBU3TJMJSGDS2SDZIE2KYYCXNT2MK4MFOQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.484911788 (56h53m5.329713311s) [noise=true , address=45.91.116.10:4082 , piece_id=C6G2APYQRU3LLXOZFU2PWQRXR3OQ2WYIADMJ76J2WVDY7HKNDTNA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.487491433 (56h53m5.332292958s) [noise=true , address=77.90.31.13:4626 , piece_id=KTVRUR4P2Z3N5CB26BLEIBORDOSZQTZM4D3E2IHEGZ4QXXWDBPXA, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.487914119 (56h53m5.332715646s) [noise=true , address=77.90.31.21:4877 , piece_id=UHPHX5KJKOB3DGWIWTMPN7P73V76GAVCOLOO4GTPMQ6BDGTCMA4A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.488256189 (56h53m5.333057714s) [noise=true , address=45.91.116.14:4191 , piece_id=M6ZKO6X2XJ7UMFZWSGNWUIEYDTXJB6C62ZXE7SVRIF6XOMCCQD4Q, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.488930410 (56h53m5.333731944s) [noise=true , address=45.91.116.6:4974 , piece_id=3W43RVQMPBJXQJWYUYX4IJ7CZUP2SWCMXZEB4AL6ITZ3OVPMZA2A, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:22.489313463 (56h53m5.334114987s) [noise=true , address=45.91.116.11:4119 , piece_id=DRIZSI6XSABNTXWUMNFY3RPV3HPUBVE62HXN7XG55FM44TMVCYSQ, seg_pos={0 0}, upload=54498]: Uploading piece... Jul 26 20:28:23.003858627 (56h53m5.848660156s) [upload=54498]: Next returned nil segment Jul 26 20:28:23.003881158 (56h53m5.848682712s) [upload=54498]: Waiting for error group managing segments... Jul 26 20:28:23.117181340 (56h53m5.961982857s) [noise=true , address=77.90.31.19:4843 , piece_id=GY3TBUVRZWDCJKIBM3GBPA2MMDPKB3VYF6CIRTWJ32MHXXNIDX7Q, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.121502780 (56h53m5.966304296s) [noise=true , address=77.90.31.15:4698 , piece_id=UZBDOABD2WFFPYJCC5EYJUZX6NU4REM5MRZF7POAQR3TXTZ5HXPQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.128715466 (56h53m5.973516987s) [noise=true , address=77.90.31.12:4589 , piece_id=VUCKZDOASEF646TUXJEEUR2IEMVPAE4LV4I57AB6HMNQG2CZCTWA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.136428815 (56h53m5.981230348s) [noise=true , address=77.90.31.14:4662 , piece_id=LFCOMWII6WAEYRC3J4NMTC2RNOS3MX4WNLDUFPI4TGPWO64AFOXQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.140913043 (56h53m5.985714576s) [noise=true , address=77.90.31.13:4626 , piece_id=KTVRUR4P2Z3N5CB26BLEIBORDOSZQTZM4D3E2IHEGZ4QXXWDBPXA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.144098231 (56h53m5.988899746s) [noise=true , address=77.90.31.21:4877 , piece_id=UHPHX5KJKOB3DGWIWTMPN7P73V76GAVCOLOO4GTPMQ6BDGTCMA4A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.154118120 (56h53m5.998919628s) [noise=true , address=77.90.31.21:4878 , piece_id=3L4TERTQQBVRHMR2IXFGYGPVDJQK6KCPS6DFOJCI2KAVG5KBLJ3A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:23.851371196 (56h53m6.696172711s) [noise=true , address=104.36.82.8:4969 , piece_id=EQKLKHNZE7DGZOBGZRBUA4BTVJU6AVHKHK7CZAXCEKTKULNG2E5A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: <nil> Jul 26 20:28:52.417946210 (56h53m35.262747746s) [noise=true , address=45.91.116.8:4047 , piece_id=QCPM7CFY5UBXWAZEAFO57NELB5R62GZ7CG2ISD2TSWAMNG6LYTRQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: rpc: dial tcp 45.91.116.8:4047: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418043876 (56h53m35.26284539s) [noise=true , address=45.91.116.14:4191 , piece_id=M6ZKO6X2XJ7UMFZWSGNWUIEYDTXJB6C62ZXE7SVRIF6XOMCCQD4Q, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418090229 (56h53m35.262891746s) [noise=true , address=45.91.116.13:4154 , piece_id=H6XMHU4LQO7T7NVPKAOBU3TJMJSGDS2SDZIE2KYYCXNT2MK4MFOQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 159.223.143.39:42688->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418225695 (56h53m35.263027225s) [noise=true , address=45.91.116.10:4082 , piece_id=C6G2APYQRU3LLXOZFU2PWQRXR3OQ2WYIADMJ76J2WVDY7HKNDTNA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: write tcp 159.223.143.39:60366->45.91.116.10:4082: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418340242 (56h53m35.263141768s) [noise=true , address=45.91.116.12:4226 , piece_id=FY47OOHAOWN4DZCL446MZKMNEZVUFH37DEFUGOKHDAYNG3QFFRNQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: write tcp 159.223.143.39:37720->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418429162 (56h53m35.263230689s) [noise=true , address=45.91.116.4:4938 , piece_id=XPEH2CKEBJUEDAZGL4W6SZYHFCOSVP6IU2OWIU44EVU46QB7TQOA, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418473168 (56h53m35.263274683s) [noise=true , address=45.91.116.6:4974 , piece_id=3W43RVQMPBJXQJWYUYX4IJ7CZUP2SWCMXZEB4AL6ITZ3OVPMZA2A, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 159.223.143.39:37724->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418564079 (56h53m35.263365602s) [noise=true , address=45.91.116.11:4119 , piece_id=DRIZSI6XSABNTXWUMNFY3RPV3HPUBVE62HXN7XG55FM44TMVCYSQ, seg_pos={0 0}, upload=54498]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:44972->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:52.418628408 (56h53m35.263429934s) [seg_pos={0 0}, upload=54498]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:52.444538177 (56h53m35.289339693s) [upload=54498]: Done uploading segments... Jul 26 20:28:52.444571403 (56h53m35.289372917s) [upload=54498]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 717.678195ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 214.707888ms FAIL (2025-07-26 21:08:11 UTC) orbiter warning: uplink: too many requests Duration: 214.804974ms core uplink us1 vivint

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)