Orbiter History

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.......X.......] duration: 99.55228ms remote: "tcp:34.202.203.47:443" target: https://www.storj.io/dcs-satellites FAIL (2025-07-22 17:01:07 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": EOF Duration: 99.58232ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [........X......] duration: 87.164131ms remote: "tcp:34.202.203.47:443" target: https://www.storj.io/dcs-satellites FAIL (2025-07-22 17:00:46 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": EOF Duration: 87.191568ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.........X.....] duration: 249.999585ms remote: "tcp:34.202.203.47:443" target: https://www.storj.io/dcs-satellites FAIL (2025-07-22 17:00:15 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": EOF Duration: 250.026741ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..........X....] duration: 111.609989ms remote: "tcp:34.202.203.47:443" target: https://www.storj.io/dcs-satellites FAIL (2025-07-22 16:59:01 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": EOF Duration: 111.671538ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [...........X...] duration: 81.750729ms remote: "tcp:34.202.203.47:443" target: https://www.storj.io/dcs-satellites FAIL (2025-07-22 16:58:36 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": EOF Duration: 81.777785ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [............X..] duration: 129.045391ms remote: "tcp:34.202.203.47:443" target: https://www.storj.io/dcs-satellites FAIL (2025-07-22 16:57:58 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": EOF Duration: 129.078847ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.............X.] duration: 4.312048715s remote: "tcp:34.202.203.47:443" target: https://www.storj.io/dcs-satellites FAIL (2025-07-22 16:57:42 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": EOF Duration: 4.312065805s core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..............X] duration: 1m4.799936611s remote: "tcp:34.202.203.47:443" target: https://www.storj.io/dcs-satellites FAIL (2025-07-22 16:57:37 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": EOF Duration: 1m4.799962749s core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 585.097811ms FAIL (2025-07-21 23:58:17 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments... Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil> Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments... Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment... Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment. Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments... Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces. Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments... Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil> Duration: 585.143412ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 410.645653ms FAIL (2025-07-21 23:57:43 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments... Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil> Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments... Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment... Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment. Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments... Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces. Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments... Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil> Duration: 410.694459ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 434.745333ms FAIL (2025-07-21 23:56:49 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments... Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil> Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments... Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment... Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment. Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments... Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces. Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments... Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil> Duration: 434.773677ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 679.532583ms FAIL (2025-07-21 23:55:48 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments... Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil> Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments... Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment... Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment. Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments... Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces. Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments... Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil> Duration: 679.560363ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 638.354255ms FAIL (2025-07-21 23:54:46 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments... Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil> Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments... Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment... Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment. Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments... Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces. Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments... Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil> Duration: 638.381384ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 560.375827ms FAIL (2025-07-21 23:53:49 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments... Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil> Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments... Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment... Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment. Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments... Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces. Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments... Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil> Duration: 560.403891ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 703.062812ms FAIL (2025-07-21 23:52:47 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments... Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil> Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments... Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment... Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment. Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments... Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces. Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments... Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil> Duration: 703.087556ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 430.580105ms FAIL (2025-07-21 23:51:49 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments... Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil> Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments... Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment... Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment. Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments... Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces. Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments... Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil> Duration: 430.611196ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 21.848172787s FAIL (2025-07-21 23:51:27 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 21.848233406s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 941.419976ms FAIL (2025-07-21 23:51:21 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments... Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil> Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments... Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment... Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment. Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments... Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces. Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments... Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil> Duration: 941.451146ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 595.694235ms FAIL (2025-07-21 23:50:31 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments... Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil> Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments... Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment... Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment. Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments... Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces. Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments... Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil> Duration: 595.729958ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 825.562954ms FAIL (2025-07-21 23:50:11 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 825.652292ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 1.319676968s FAIL (2025-07-21 23:49:37 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments... Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil> Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments... Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment... Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment. Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments... Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces. Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments... Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil> Duration: 1.319711039s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 4.389324481s FAIL (2025-07-21 23:49:18 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.389399721s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [X..X........................................................] duration: 10.541937607s FAIL (2025-07-21 23:49:04 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 10.541958958s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 1.400073074s FAIL (2025-07-21 23:48:50 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments... Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil> Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments... Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment... Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment. Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments... Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces. Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments... Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil> Duration: 1.400100776s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.X..X.......................................................] duration: 16.687000691s FAIL (2025-07-21 23:48:10 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 16.68702536s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 954.179927ms FAIL (2025-07-21 23:48:02 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments... Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil> Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments... Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment... Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment. Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments... Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces. Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments... Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil> Duration: 954.210797ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 672.834589ms FAIL (2025-07-21 23:47:42 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments... Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil> Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments... Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment... Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment. Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments... Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces. Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments... Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil> Duration: 672.864906ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..X..X......................................................] duration: 9.165484325s FAIL (2025-07-21 23:47:11 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 9.165533268s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 710.495897ms FAIL (2025-07-21 23:46:54 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments... Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil> Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments... Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment... Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment. Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments... Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces. Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments... Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil> Duration: 710.52715ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...X..X.....................................................] duration: 24.506068301s FAIL (2025-07-21 23:46:24 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 24.506105258s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 845.516136ms FAIL (2025-07-21 23:46:12 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments... Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil> Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments... Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment... Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment. Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments... Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces. Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments... Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil> Duration: 845.539618ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 508.697033ms FAIL (2025-07-21 23:45:47 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments... Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil> Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments... Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment... Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment. Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments... Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces. Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments... Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil> Duration: 508.723921ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [....X..X....................................................] duration: 2.039405107s FAIL (2025-07-21 23:45:13 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.0394287s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 453.135426ms FAIL (2025-07-21 23:44:54 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments... Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil> Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments... Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment... Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment. Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments... Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces. Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments... Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil> Duration: 453.156786ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [X....X..X...................................................] duration: 16.420654751s FAIL (2025-07-21 23:44:13 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 16.420692379s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.7166619s FAIL (2025-07-21 23:43:51 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments... Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil> Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments... Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment... Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment. Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments... Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces. Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments... Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil> Duration: 2.716931737s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 3.127893186s FAIL (2025-07-21 23:43:30 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments... Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil> Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments... Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment... Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment. Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments... Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces. Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments... Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil> Duration: 3.127929136s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.X....X..X..................................................] duration: 16.202952899s FAIL (2025-07-21 23:43:12 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 16.202973402s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 502.970019ms FAIL (2025-07-21 23:42:39 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments... Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil> Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments... Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment... Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment. Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments... Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces. Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments... Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil> Duration: 503.020398ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [..X....X..X.................................................] duration: 7.962425546s FAIL (2025-07-21 23:42:15 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.962599201s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 1.208338952s FAIL (2025-07-21 23:41:53 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments... Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil> Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments... Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment... Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment. Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments... Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces. Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments... Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil> Duration: 1.208366291s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [...X....X..X................................................] duration: 15.640311376s FAIL (2025-07-21 23:41:26 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 15.640338381s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 2.927269855s FAIL (2025-07-21 23:40:42 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments... Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil> Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments... Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment... Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment. Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments... Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces. Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments... Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil> Duration: 2.927298393s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [....X....X..X...............................................] duration: 7.934300525s FAIL (2025-07-21 23:40:36 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.934340286s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.....X....X..X..............................................] duration: 9.965630125s FAIL (2025-07-21 23:39:50 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 9.965654258s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 616.465727ms FAIL (2025-07-21 23:39:40 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments... Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil> Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments... Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment... Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment. Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments... Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces. Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments... Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil> Duration: 616.492251ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [X.....X....X..X.............................................] duration: 8.087634709s FAIL (2025-07-21 23:39:02 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.087659057s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 4.819202158s FAIL (2025-07-21 23:38:48 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments... Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil> Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments... Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment... Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment. Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments... Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces. Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments... Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil> Duration: 4.819245318s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 462.201663ms FAIL (2025-07-21 23:38:10 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments... Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil> Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments... Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment... Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment. Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece... Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments... Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil> Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces. Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments... Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil> Duration: 462.232928ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.X.....X....X..X............................................] duration: 15.819114069s FAIL (2025-07-21 23:38:08 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 15.819145007s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)