Orbiter History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.412104774s trace: https://jaeger.dev.storj.io/trace/12dbee1725a89af6 FAIL (2025-12-13 13:39:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=197386]: Uploading piece... Dec 13 13:20:22.157523997 (189h52m22.985232392s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157563526 (189h52m22.985271946s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157782226 (189h52m22.985490616s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158220136 (189h52m22.98592853s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158561593 (189h52m22.986269971s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158833133 (189h52m22.986541524s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159110305 (189h52m22.986818699s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159379461 (189h52m22.987087838s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159627778 (189h52m22.987336156s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.588165740 (189h52m23.41587413s) [upload=197386]: Next returned nil segment Dec 13 13:20:22.588188840 (189h52m23.415897231s) [upload=197386]: Waiting for error group managing segments... Dec 13 13:20:22.787355936 (189h52m23.615064332s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.793561088 (189h52m23.621269481s) [noise=true , address=185.167.204.205:4013 , piece_id=QNJLAFTMBQPQF7IBMT6EKMBEL5KX7VHU3L4L3ALPXYNUV2MZ43GA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.801170609 (189h52m23.628879002s) [noise=true , address=185.167.204.206:4068 , piece_id=P2FTDV3XW34Z6M73CWIURQCCQ6H5NXDNJWKBVN3GRQTDV65QLP5A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.812199494 (189h52m23.63990789s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813259194 (189h52m23.640967588s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813831001 (189h52m23.641539395s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814310957 (189h52m23.642019347s) [noise=true , address=185.167.204.205:4001 , piece_id=WXEZMLB4ZM2FFOB4JMCIF2X6FTB72I3EWWRTDLIZ6SXCMUTS5HGQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814961478 (189h52m23.64266987s) [noise=true , address=185.167.204.206:4046 , piece_id=KLHWCCQLK3CAOIMW57WAFZHNMECYLGA4VRZE3LXRUJRQCKAGGBKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.815469120 (189h52m23.643177512s) [noise=true , address=185.167.204.206:4054 , piece_id=ZPZSVEBAXMRI7732XKIHUGMU2WGGCNKE34U2FZ3NHJLKCMYBIUXQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.817780190 (189h52m23.645488584s) [noise=true , address=185.167.204.206:4045 , piece_id=VQ37X765RRBAVPI4HIAPJET7VRCI2FE7FRHB45II6ETFYDYMKSTQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818186706 (189h52m23.645895101s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818345893 (189h52m23.646054284s) [noise=true , address=185.167.204.205:4014 , piece_id=4U6VXOGIXKAV7NJBQBD2V6ZVWAUIT557OP7AUG332UWVS5LZFSPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.844792090 (189h52m23.672516847s) [noise=true , address=185.167.204.206:4051 , piece_id=A4VZAGPIVLUVOVOIGGIK7CEDRPUSV34XMUL5TFL5KFJNICAWSJ4Q, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.354540799 (189h52m25.182249185s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.369904036 (189h52m25.197612417s) [noise=true , address=81.171.14.70:4039 , piece_id=YV7FP35KY6TXYDKBEOV6RCHCSJWR4TQCFEB43GK3AZP4C776BCZQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.378758619 (189h52m25.206467012s) [noise=true , address=81.171.14.70:4047 , piece_id=EQFLTLGTZBKQ47MWFLSBCEHDKAWMZSGJHZPBAN67EANSP2LR7HYQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.384761037 (189h52m25.212469431s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.495107931 (189h52m25.322816323s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.833114410 (189h52m25.660822801s) [noise=true , address=89.149.201.147:4009 , piece_id=SAES2PXHWVMC22L3DKLBYHAI3ZHKYKGYOULL4ZZ2AB2U53B5URVA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.076006180 (189h52m25.903714572s) [noise=true , address=89.149.201.147:4008 , piece_id=HL2EBKXU5RWRNZP7D3MUOTRJK2YJTADFQS6C4IF7CGEUF3PLOIMQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.109256616 (189h52m25.936964999s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.121296573 (189h52m25.949004965s) [noise=true , address=81.171.14.70:4034 , piece_id=XZ72ODCL37AYLPMMFTCT4NJROVRCLTEM2XN6SQVG3G5NTZRJDWDA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.144861937 (189h52m25.972570318s) [noise=true , address=89.149.201.147:4012 , piece_id=5LDACT5ZGR54I4LNLZQHNXUY2VZQ5XIWI5AYUM5CZHCOOQIBGPRQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.859441967 (189h52m26.687150358s) [noise=true , address=81.171.14.70:4026 , piece_id=R7B2VE64HYLI7A2XVWDFTIIWAVLT6I6E75ZCAMTDH77R7BW2Q7IQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.932179545 (189h52m26.759887944s) [noise=true , address=89.149.201.147:4001 , piece_id=GDNVHYCJJDXLUDF77L6BP4IMIMR6KRZ2FTZJ6BG5TMAZGIW75PHQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228904886 (189h52m27.056613283s) [noise=true , address=89.149.201.147:4024 , piece_id=7MCMBHNW4L7JDB4ATRHAM3GWVFAYH34LWFNX5O7C6EXIB34BNWRA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228954088 (189h52m27.056662491s) [seg_pos={0 0}, upload=197386]: Segment reached optimal threshold of 26 pieces. Dec 13 13:20:26.229843002 (189h52m27.057551404s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: read tcp 159.223.143.39:53228->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.229857857 (189h52m27.05756626s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230208353 (189h52m27.057916754s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7): protocol: read tcp 159.223.143.39:45318->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.230219436 (189h52m27.057927837s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230249796 (189h52m27.057958176s) [seg_pos={0 0}, upload=197386]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 13 13:20:26.230536147 (189h52m27.05824454s) [upload=197386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 13 13:20:26.230543508 (189h52m27.058251899s) [upload=197386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 13 13:20:26.379840216 (189h52m27.207548608s) [upload=197386]: Done uploading segments... Dec 13 13:20:26.379861196 (189h52m27.207569587s) [upload=197386]: Upload finished. err: <nil> Duration: 5.412152721s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 3.982689282s trace: https://jaeger.dev.storj.io/trace/12dbee1725a89af6 FAIL (2025-12-13 13:38:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=197386]: Uploading piece... Dec 13 13:20:22.157523997 (189h52m22.985232392s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157563526 (189h52m22.985271946s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157782226 (189h52m22.985490616s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158220136 (189h52m22.98592853s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158561593 (189h52m22.986269971s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158833133 (189h52m22.986541524s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159110305 (189h52m22.986818699s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159379461 (189h52m22.987087838s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159627778 (189h52m22.987336156s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.588165740 (189h52m23.41587413s) [upload=197386]: Next returned nil segment Dec 13 13:20:22.588188840 (189h52m23.415897231s) [upload=197386]: Waiting for error group managing segments... Dec 13 13:20:22.787355936 (189h52m23.615064332s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.793561088 (189h52m23.621269481s) [noise=true , address=185.167.204.205:4013 , piece_id=QNJLAFTMBQPQF7IBMT6EKMBEL5KX7VHU3L4L3ALPXYNUV2MZ43GA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.801170609 (189h52m23.628879002s) [noise=true , address=185.167.204.206:4068 , piece_id=P2FTDV3XW34Z6M73CWIURQCCQ6H5NXDNJWKBVN3GRQTDV65QLP5A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.812199494 (189h52m23.63990789s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813259194 (189h52m23.640967588s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813831001 (189h52m23.641539395s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814310957 (189h52m23.642019347s) [noise=true , address=185.167.204.205:4001 , piece_id=WXEZMLB4ZM2FFOB4JMCIF2X6FTB72I3EWWRTDLIZ6SXCMUTS5HGQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814961478 (189h52m23.64266987s) [noise=true , address=185.167.204.206:4046 , piece_id=KLHWCCQLK3CAOIMW57WAFZHNMECYLGA4VRZE3LXRUJRQCKAGGBKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.815469120 (189h52m23.643177512s) [noise=true , address=185.167.204.206:4054 , piece_id=ZPZSVEBAXMRI7732XKIHUGMU2WGGCNKE34U2FZ3NHJLKCMYBIUXQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.817780190 (189h52m23.645488584s) [noise=true , address=185.167.204.206:4045 , piece_id=VQ37X765RRBAVPI4HIAPJET7VRCI2FE7FRHB45II6ETFYDYMKSTQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818186706 (189h52m23.645895101s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818345893 (189h52m23.646054284s) [noise=true , address=185.167.204.205:4014 , piece_id=4U6VXOGIXKAV7NJBQBD2V6ZVWAUIT557OP7AUG332UWVS5LZFSPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.844792090 (189h52m23.672516847s) [noise=true , address=185.167.204.206:4051 , piece_id=A4VZAGPIVLUVOVOIGGIK7CEDRPUSV34XMUL5TFL5KFJNICAWSJ4Q, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.354540799 (189h52m25.182249185s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.369904036 (189h52m25.197612417s) [noise=true , address=81.171.14.70:4039 , piece_id=YV7FP35KY6TXYDKBEOV6RCHCSJWR4TQCFEB43GK3AZP4C776BCZQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.378758619 (189h52m25.206467012s) [noise=true , address=81.171.14.70:4047 , piece_id=EQFLTLGTZBKQ47MWFLSBCEHDKAWMZSGJHZPBAN67EANSP2LR7HYQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.384761037 (189h52m25.212469431s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.495107931 (189h52m25.322816323s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.833114410 (189h52m25.660822801s) [noise=true , address=89.149.201.147:4009 , piece_id=SAES2PXHWVMC22L3DKLBYHAI3ZHKYKGYOULL4ZZ2AB2U53B5URVA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.076006180 (189h52m25.903714572s) [noise=true , address=89.149.201.147:4008 , piece_id=HL2EBKXU5RWRNZP7D3MUOTRJK2YJTADFQS6C4IF7CGEUF3PLOIMQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.109256616 (189h52m25.936964999s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.121296573 (189h52m25.949004965s) [noise=true , address=81.171.14.70:4034 , piece_id=XZ72ODCL37AYLPMMFTCT4NJROVRCLTEM2XN6SQVG3G5NTZRJDWDA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.144861937 (189h52m25.972570318s) [noise=true , address=89.149.201.147:4012 , piece_id=5LDACT5ZGR54I4LNLZQHNXUY2VZQ5XIWI5AYUM5CZHCOOQIBGPRQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.859441967 (189h52m26.687150358s) [noise=true , address=81.171.14.70:4026 , piece_id=R7B2VE64HYLI7A2XVWDFTIIWAVLT6I6E75ZCAMTDH77R7BW2Q7IQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.932179545 (189h52m26.759887944s) [noise=true , address=89.149.201.147:4001 , piece_id=GDNVHYCJJDXLUDF77L6BP4IMIMR6KRZ2FTZJ6BG5TMAZGIW75PHQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228904886 (189h52m27.056613283s) [noise=true , address=89.149.201.147:4024 , piece_id=7MCMBHNW4L7JDB4ATRHAM3GWVFAYH34LWFNX5O7C6EXIB34BNWRA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228954088 (189h52m27.056662491s) [seg_pos={0 0}, upload=197386]: Segment reached optimal threshold of 26 pieces. Dec 13 13:20:26.229843002 (189h52m27.057551404s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: read tcp 159.223.143.39:53228->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.229857857 (189h52m27.05756626s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230208353 (189h52m27.057916754s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7): protocol: read tcp 159.223.143.39:45318->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.230219436 (189h52m27.057927837s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230249796 (189h52m27.057958176s) [seg_pos={0 0}, upload=197386]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 13 13:20:26.230536147 (189h52m27.05824454s) [upload=197386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 13 13:20:26.230543508 (189h52m27.058251899s) [upload=197386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 13 13:20:26.379840216 (189h52m27.207548608s) [upload=197386]: Done uploading segments... Dec 13 13:20:26.379861196 (189h52m27.207569587s) [upload=197386]: Upload finished. err: <nil> Duration: 3.982729964s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 17.002422321s trace: https://jaeger.dev.storj.io/trace/12dbee1725a89af6 FAIL (2025-12-13 13:37:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=197386]: Uploading piece... Dec 13 13:20:22.157523997 (189h52m22.985232392s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157563526 (189h52m22.985271946s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157782226 (189h52m22.985490616s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158220136 (189h52m22.98592853s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158561593 (189h52m22.986269971s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158833133 (189h52m22.986541524s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159110305 (189h52m22.986818699s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159379461 (189h52m22.987087838s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159627778 (189h52m22.987336156s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.588165740 (189h52m23.41587413s) [upload=197386]: Next returned nil segment Dec 13 13:20:22.588188840 (189h52m23.415897231s) [upload=197386]: Waiting for error group managing segments... Dec 13 13:20:22.787355936 (189h52m23.615064332s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.793561088 (189h52m23.621269481s) [noise=true , address=185.167.204.205:4013 , piece_id=QNJLAFTMBQPQF7IBMT6EKMBEL5KX7VHU3L4L3ALPXYNUV2MZ43GA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.801170609 (189h52m23.628879002s) [noise=true , address=185.167.204.206:4068 , piece_id=P2FTDV3XW34Z6M73CWIURQCCQ6H5NXDNJWKBVN3GRQTDV65QLP5A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.812199494 (189h52m23.63990789s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813259194 (189h52m23.640967588s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813831001 (189h52m23.641539395s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814310957 (189h52m23.642019347s) [noise=true , address=185.167.204.205:4001 , piece_id=WXEZMLB4ZM2FFOB4JMCIF2X6FTB72I3EWWRTDLIZ6SXCMUTS5HGQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814961478 (189h52m23.64266987s) [noise=true , address=185.167.204.206:4046 , piece_id=KLHWCCQLK3CAOIMW57WAFZHNMECYLGA4VRZE3LXRUJRQCKAGGBKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.815469120 (189h52m23.643177512s) [noise=true , address=185.167.204.206:4054 , piece_id=ZPZSVEBAXMRI7732XKIHUGMU2WGGCNKE34U2FZ3NHJLKCMYBIUXQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.817780190 (189h52m23.645488584s) [noise=true , address=185.167.204.206:4045 , piece_id=VQ37X765RRBAVPI4HIAPJET7VRCI2FE7FRHB45II6ETFYDYMKSTQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818186706 (189h52m23.645895101s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818345893 (189h52m23.646054284s) [noise=true , address=185.167.204.205:4014 , piece_id=4U6VXOGIXKAV7NJBQBD2V6ZVWAUIT557OP7AUG332UWVS5LZFSPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.844792090 (189h52m23.672516847s) [noise=true , address=185.167.204.206:4051 , piece_id=A4VZAGPIVLUVOVOIGGIK7CEDRPUSV34XMUL5TFL5KFJNICAWSJ4Q, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.354540799 (189h52m25.182249185s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.369904036 (189h52m25.197612417s) [noise=true , address=81.171.14.70:4039 , piece_id=YV7FP35KY6TXYDKBEOV6RCHCSJWR4TQCFEB43GK3AZP4C776BCZQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.378758619 (189h52m25.206467012s) [noise=true , address=81.171.14.70:4047 , piece_id=EQFLTLGTZBKQ47MWFLSBCEHDKAWMZSGJHZPBAN67EANSP2LR7HYQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.384761037 (189h52m25.212469431s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.495107931 (189h52m25.322816323s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.833114410 (189h52m25.660822801s) [noise=true , address=89.149.201.147:4009 , piece_id=SAES2PXHWVMC22L3DKLBYHAI3ZHKYKGYOULL4ZZ2AB2U53B5URVA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.076006180 (189h52m25.903714572s) [noise=true , address=89.149.201.147:4008 , piece_id=HL2EBKXU5RWRNZP7D3MUOTRJK2YJTADFQS6C4IF7CGEUF3PLOIMQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.109256616 (189h52m25.936964999s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.121296573 (189h52m25.949004965s) [noise=true , address=81.171.14.70:4034 , piece_id=XZ72ODCL37AYLPMMFTCT4NJROVRCLTEM2XN6SQVG3G5NTZRJDWDA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.144861937 (189h52m25.972570318s) [noise=true , address=89.149.201.147:4012 , piece_id=5LDACT5ZGR54I4LNLZQHNXUY2VZQ5XIWI5AYUM5CZHCOOQIBGPRQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.859441967 (189h52m26.687150358s) [noise=true , address=81.171.14.70:4026 , piece_id=R7B2VE64HYLI7A2XVWDFTIIWAVLT6I6E75ZCAMTDH77R7BW2Q7IQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.932179545 (189h52m26.759887944s) [noise=true , address=89.149.201.147:4001 , piece_id=GDNVHYCJJDXLUDF77L6BP4IMIMR6KRZ2FTZJ6BG5TMAZGIW75PHQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228904886 (189h52m27.056613283s) [noise=true , address=89.149.201.147:4024 , piece_id=7MCMBHNW4L7JDB4ATRHAM3GWVFAYH34LWFNX5O7C6EXIB34BNWRA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228954088 (189h52m27.056662491s) [seg_pos={0 0}, upload=197386]: Segment reached optimal threshold of 26 pieces. Dec 13 13:20:26.229843002 (189h52m27.057551404s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: read tcp 159.223.143.39:53228->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.229857857 (189h52m27.05756626s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230208353 (189h52m27.057916754s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7): protocol: read tcp 159.223.143.39:45318->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.230219436 (189h52m27.057927837s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230249796 (189h52m27.057958176s) [seg_pos={0 0}, upload=197386]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 13 13:20:26.230536147 (189h52m27.05824454s) [upload=197386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 13 13:20:26.230543508 (189h52m27.058251899s) [upload=197386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 13 13:20:26.379840216 (189h52m27.207548608s) [upload=197386]: Done uploading segments... Dec 13 13:20:26.379861196 (189h52m27.207569587s) [upload=197386]: Upload finished. err: <nil> Duration: 17.002447567s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 3.751949862s trace: https://jaeger.dev.storj.io/trace/12dbee1725a89af6 FAIL (2025-12-13 13:37:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=197386]: Uploading piece... Dec 13 13:20:22.157523997 (189h52m22.985232392s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157563526 (189h52m22.985271946s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157782226 (189h52m22.985490616s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158220136 (189h52m22.98592853s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158561593 (189h52m22.986269971s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158833133 (189h52m22.986541524s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159110305 (189h52m22.986818699s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159379461 (189h52m22.987087838s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159627778 (189h52m22.987336156s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.588165740 (189h52m23.41587413s) [upload=197386]: Next returned nil segment Dec 13 13:20:22.588188840 (189h52m23.415897231s) [upload=197386]: Waiting for error group managing segments... Dec 13 13:20:22.787355936 (189h52m23.615064332s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.793561088 (189h52m23.621269481s) [noise=true , address=185.167.204.205:4013 , piece_id=QNJLAFTMBQPQF7IBMT6EKMBEL5KX7VHU3L4L3ALPXYNUV2MZ43GA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.801170609 (189h52m23.628879002s) [noise=true , address=185.167.204.206:4068 , piece_id=P2FTDV3XW34Z6M73CWIURQCCQ6H5NXDNJWKBVN3GRQTDV65QLP5A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.812199494 (189h52m23.63990789s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813259194 (189h52m23.640967588s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813831001 (189h52m23.641539395s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814310957 (189h52m23.642019347s) [noise=true , address=185.167.204.205:4001 , piece_id=WXEZMLB4ZM2FFOB4JMCIF2X6FTB72I3EWWRTDLIZ6SXCMUTS5HGQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814961478 (189h52m23.64266987s) [noise=true , address=185.167.204.206:4046 , piece_id=KLHWCCQLK3CAOIMW57WAFZHNMECYLGA4VRZE3LXRUJRQCKAGGBKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.815469120 (189h52m23.643177512s) [noise=true , address=185.167.204.206:4054 , piece_id=ZPZSVEBAXMRI7732XKIHUGMU2WGGCNKE34U2FZ3NHJLKCMYBIUXQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.817780190 (189h52m23.645488584s) [noise=true , address=185.167.204.206:4045 , piece_id=VQ37X765RRBAVPI4HIAPJET7VRCI2FE7FRHB45II6ETFYDYMKSTQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818186706 (189h52m23.645895101s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818345893 (189h52m23.646054284s) [noise=true , address=185.167.204.205:4014 , piece_id=4U6VXOGIXKAV7NJBQBD2V6ZVWAUIT557OP7AUG332UWVS5LZFSPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.844792090 (189h52m23.672516847s) [noise=true , address=185.167.204.206:4051 , piece_id=A4VZAGPIVLUVOVOIGGIK7CEDRPUSV34XMUL5TFL5KFJNICAWSJ4Q, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.354540799 (189h52m25.182249185s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.369904036 (189h52m25.197612417s) [noise=true , address=81.171.14.70:4039 , piece_id=YV7FP35KY6TXYDKBEOV6RCHCSJWR4TQCFEB43GK3AZP4C776BCZQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.378758619 (189h52m25.206467012s) [noise=true , address=81.171.14.70:4047 , piece_id=EQFLTLGTZBKQ47MWFLSBCEHDKAWMZSGJHZPBAN67EANSP2LR7HYQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.384761037 (189h52m25.212469431s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.495107931 (189h52m25.322816323s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.833114410 (189h52m25.660822801s) [noise=true , address=89.149.201.147:4009 , piece_id=SAES2PXHWVMC22L3DKLBYHAI3ZHKYKGYOULL4ZZ2AB2U53B5URVA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.076006180 (189h52m25.903714572s) [noise=true , address=89.149.201.147:4008 , piece_id=HL2EBKXU5RWRNZP7D3MUOTRJK2YJTADFQS6C4IF7CGEUF3PLOIMQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.109256616 (189h52m25.936964999s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.121296573 (189h52m25.949004965s) [noise=true , address=81.171.14.70:4034 , piece_id=XZ72ODCL37AYLPMMFTCT4NJROVRCLTEM2XN6SQVG3G5NTZRJDWDA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.144861937 (189h52m25.972570318s) [noise=true , address=89.149.201.147:4012 , piece_id=5LDACT5ZGR54I4LNLZQHNXUY2VZQ5XIWI5AYUM5CZHCOOQIBGPRQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.859441967 (189h52m26.687150358s) [noise=true , address=81.171.14.70:4026 , piece_id=R7B2VE64HYLI7A2XVWDFTIIWAVLT6I6E75ZCAMTDH77R7BW2Q7IQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.932179545 (189h52m26.759887944s) [noise=true , address=89.149.201.147:4001 , piece_id=GDNVHYCJJDXLUDF77L6BP4IMIMR6KRZ2FTZJ6BG5TMAZGIW75PHQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228904886 (189h52m27.056613283s) [noise=true , address=89.149.201.147:4024 , piece_id=7MCMBHNW4L7JDB4ATRHAM3GWVFAYH34LWFNX5O7C6EXIB34BNWRA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228954088 (189h52m27.056662491s) [seg_pos={0 0}, upload=197386]: Segment reached optimal threshold of 26 pieces. Dec 13 13:20:26.229843002 (189h52m27.057551404s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: read tcp 159.223.143.39:53228->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.229857857 (189h52m27.05756626s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230208353 (189h52m27.057916754s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7): protocol: read tcp 159.223.143.39:45318->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.230219436 (189h52m27.057927837s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230249796 (189h52m27.057958176s) [seg_pos={0 0}, upload=197386]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 13 13:20:26.230536147 (189h52m27.05824454s) [upload=197386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 13 13:20:26.230543508 (189h52m27.058251899s) [upload=197386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 13 13:20:26.379840216 (189h52m27.207548608s) [upload=197386]: Done uploading segments... Dec 13 13:20:26.379861196 (189h52m27.207569587s) [upload=197386]: Upload finished. err: <nil> Duration: 3.75204759s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 3.197489323s trace: https://jaeger.dev.storj.io/trace/12dbee1725a89af6 FAIL (2025-12-13 13:36:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=197386]: Uploading piece... Dec 13 13:20:22.157523997 (189h52m22.985232392s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157563526 (189h52m22.985271946s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157782226 (189h52m22.985490616s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158220136 (189h52m22.98592853s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158561593 (189h52m22.986269971s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158833133 (189h52m22.986541524s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159110305 (189h52m22.986818699s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159379461 (189h52m22.987087838s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159627778 (189h52m22.987336156s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.588165740 (189h52m23.41587413s) [upload=197386]: Next returned nil segment Dec 13 13:20:22.588188840 (189h52m23.415897231s) [upload=197386]: Waiting for error group managing segments... Dec 13 13:20:22.787355936 (189h52m23.615064332s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.793561088 (189h52m23.621269481s) [noise=true , address=185.167.204.205:4013 , piece_id=QNJLAFTMBQPQF7IBMT6EKMBEL5KX7VHU3L4L3ALPXYNUV2MZ43GA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.801170609 (189h52m23.628879002s) [noise=true , address=185.167.204.206:4068 , piece_id=P2FTDV3XW34Z6M73CWIURQCCQ6H5NXDNJWKBVN3GRQTDV65QLP5A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.812199494 (189h52m23.63990789s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813259194 (189h52m23.640967588s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813831001 (189h52m23.641539395s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814310957 (189h52m23.642019347s) [noise=true , address=185.167.204.205:4001 , piece_id=WXEZMLB4ZM2FFOB4JMCIF2X6FTB72I3EWWRTDLIZ6SXCMUTS5HGQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814961478 (189h52m23.64266987s) [noise=true , address=185.167.204.206:4046 , piece_id=KLHWCCQLK3CAOIMW57WAFZHNMECYLGA4VRZE3LXRUJRQCKAGGBKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.815469120 (189h52m23.643177512s) [noise=true , address=185.167.204.206:4054 , piece_id=ZPZSVEBAXMRI7732XKIHUGMU2WGGCNKE34U2FZ3NHJLKCMYBIUXQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.817780190 (189h52m23.645488584s) [noise=true , address=185.167.204.206:4045 , piece_id=VQ37X765RRBAVPI4HIAPJET7VRCI2FE7FRHB45II6ETFYDYMKSTQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818186706 (189h52m23.645895101s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818345893 (189h52m23.646054284s) [noise=true , address=185.167.204.205:4014 , piece_id=4U6VXOGIXKAV7NJBQBD2V6ZVWAUIT557OP7AUG332UWVS5LZFSPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.844792090 (189h52m23.672516847s) [noise=true , address=185.167.204.206:4051 , piece_id=A4VZAGPIVLUVOVOIGGIK7CEDRPUSV34XMUL5TFL5KFJNICAWSJ4Q, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.354540799 (189h52m25.182249185s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.369904036 (189h52m25.197612417s) [noise=true , address=81.171.14.70:4039 , piece_id=YV7FP35KY6TXYDKBEOV6RCHCSJWR4TQCFEB43GK3AZP4C776BCZQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.378758619 (189h52m25.206467012s) [noise=true , address=81.171.14.70:4047 , piece_id=EQFLTLGTZBKQ47MWFLSBCEHDKAWMZSGJHZPBAN67EANSP2LR7HYQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.384761037 (189h52m25.212469431s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.495107931 (189h52m25.322816323s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.833114410 (189h52m25.660822801s) [noise=true , address=89.149.201.147:4009 , piece_id=SAES2PXHWVMC22L3DKLBYHAI3ZHKYKGYOULL4ZZ2AB2U53B5URVA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.076006180 (189h52m25.903714572s) [noise=true , address=89.149.201.147:4008 , piece_id=HL2EBKXU5RWRNZP7D3MUOTRJK2YJTADFQS6C4IF7CGEUF3PLOIMQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.109256616 (189h52m25.936964999s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.121296573 (189h52m25.949004965s) [noise=true , address=81.171.14.70:4034 , piece_id=XZ72ODCL37AYLPMMFTCT4NJROVRCLTEM2XN6SQVG3G5NTZRJDWDA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.144861937 (189h52m25.972570318s) [noise=true , address=89.149.201.147:4012 , piece_id=5LDACT5ZGR54I4LNLZQHNXUY2VZQ5XIWI5AYUM5CZHCOOQIBGPRQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.859441967 (189h52m26.687150358s) [noise=true , address=81.171.14.70:4026 , piece_id=R7B2VE64HYLI7A2XVWDFTIIWAVLT6I6E75ZCAMTDH77R7BW2Q7IQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.932179545 (189h52m26.759887944s) [noise=true , address=89.149.201.147:4001 , piece_id=GDNVHYCJJDXLUDF77L6BP4IMIMR6KRZ2FTZJ6BG5TMAZGIW75PHQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228904886 (189h52m27.056613283s) [noise=true , address=89.149.201.147:4024 , piece_id=7MCMBHNW4L7JDB4ATRHAM3GWVFAYH34LWFNX5O7C6EXIB34BNWRA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228954088 (189h52m27.056662491s) [seg_pos={0 0}, upload=197386]: Segment reached optimal threshold of 26 pieces. Dec 13 13:20:26.229843002 (189h52m27.057551404s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: read tcp 159.223.143.39:53228->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.229857857 (189h52m27.05756626s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230208353 (189h52m27.057916754s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7): protocol: read tcp 159.223.143.39:45318->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.230219436 (189h52m27.057927837s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230249796 (189h52m27.057958176s) [seg_pos={0 0}, upload=197386]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 13 13:20:26.230536147 (189h52m27.05824454s) [upload=197386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 13 13:20:26.230543508 (189h52m27.058251899s) [upload=197386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 13 13:20:26.379840216 (189h52m27.207548608s) [upload=197386]: Done uploading segments... Dec 13 13:20:26.379861196 (189h52m27.207569587s) [upload=197386]: Upload finished. err: <nil> Duration: 3.197517876s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 3.69086246s trace: https://jaeger.dev.storj.io/trace/12dbee1725a89af6 FAIL (2025-12-13 13:36:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=197386]: Uploading piece... Dec 13 13:20:22.157523997 (189h52m22.985232392s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157563526 (189h52m22.985271946s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157782226 (189h52m22.985490616s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158220136 (189h52m22.98592853s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158561593 (189h52m22.986269971s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158833133 (189h52m22.986541524s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159110305 (189h52m22.986818699s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159379461 (189h52m22.987087838s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159627778 (189h52m22.987336156s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.588165740 (189h52m23.41587413s) [upload=197386]: Next returned nil segment Dec 13 13:20:22.588188840 (189h52m23.415897231s) [upload=197386]: Waiting for error group managing segments... Dec 13 13:20:22.787355936 (189h52m23.615064332s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.793561088 (189h52m23.621269481s) [noise=true , address=185.167.204.205:4013 , piece_id=QNJLAFTMBQPQF7IBMT6EKMBEL5KX7VHU3L4L3ALPXYNUV2MZ43GA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.801170609 (189h52m23.628879002s) [noise=true , address=185.167.204.206:4068 , piece_id=P2FTDV3XW34Z6M73CWIURQCCQ6H5NXDNJWKBVN3GRQTDV65QLP5A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.812199494 (189h52m23.63990789s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813259194 (189h52m23.640967588s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813831001 (189h52m23.641539395s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814310957 (189h52m23.642019347s) [noise=true , address=185.167.204.205:4001 , piece_id=WXEZMLB4ZM2FFOB4JMCIF2X6FTB72I3EWWRTDLIZ6SXCMUTS5HGQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814961478 (189h52m23.64266987s) [noise=true , address=185.167.204.206:4046 , piece_id=KLHWCCQLK3CAOIMW57WAFZHNMECYLGA4VRZE3LXRUJRQCKAGGBKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.815469120 (189h52m23.643177512s) [noise=true , address=185.167.204.206:4054 , piece_id=ZPZSVEBAXMRI7732XKIHUGMU2WGGCNKE34U2FZ3NHJLKCMYBIUXQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.817780190 (189h52m23.645488584s) [noise=true , address=185.167.204.206:4045 , piece_id=VQ37X765RRBAVPI4HIAPJET7VRCI2FE7FRHB45II6ETFYDYMKSTQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818186706 (189h52m23.645895101s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818345893 (189h52m23.646054284s) [noise=true , address=185.167.204.205:4014 , piece_id=4U6VXOGIXKAV7NJBQBD2V6ZVWAUIT557OP7AUG332UWVS5LZFSPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.844792090 (189h52m23.672516847s) [noise=true , address=185.167.204.206:4051 , piece_id=A4VZAGPIVLUVOVOIGGIK7CEDRPUSV34XMUL5TFL5KFJNICAWSJ4Q, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.354540799 (189h52m25.182249185s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.369904036 (189h52m25.197612417s) [noise=true , address=81.171.14.70:4039 , piece_id=YV7FP35KY6TXYDKBEOV6RCHCSJWR4TQCFEB43GK3AZP4C776BCZQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.378758619 (189h52m25.206467012s) [noise=true , address=81.171.14.70:4047 , piece_id=EQFLTLGTZBKQ47MWFLSBCEHDKAWMZSGJHZPBAN67EANSP2LR7HYQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.384761037 (189h52m25.212469431s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.495107931 (189h52m25.322816323s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.833114410 (189h52m25.660822801s) [noise=true , address=89.149.201.147:4009 , piece_id=SAES2PXHWVMC22L3DKLBYHAI3ZHKYKGYOULL4ZZ2AB2U53B5URVA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.076006180 (189h52m25.903714572s) [noise=true , address=89.149.201.147:4008 , piece_id=HL2EBKXU5RWRNZP7D3MUOTRJK2YJTADFQS6C4IF7CGEUF3PLOIMQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.109256616 (189h52m25.936964999s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.121296573 (189h52m25.949004965s) [noise=true , address=81.171.14.70:4034 , piece_id=XZ72ODCL37AYLPMMFTCT4NJROVRCLTEM2XN6SQVG3G5NTZRJDWDA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.144861937 (189h52m25.972570318s) [noise=true , address=89.149.201.147:4012 , piece_id=5LDACT5ZGR54I4LNLZQHNXUY2VZQ5XIWI5AYUM5CZHCOOQIBGPRQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.859441967 (189h52m26.687150358s) [noise=true , address=81.171.14.70:4026 , piece_id=R7B2VE64HYLI7A2XVWDFTIIWAVLT6I6E75ZCAMTDH77R7BW2Q7IQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.932179545 (189h52m26.759887944s) [noise=true , address=89.149.201.147:4001 , piece_id=GDNVHYCJJDXLUDF77L6BP4IMIMR6KRZ2FTZJ6BG5TMAZGIW75PHQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228904886 (189h52m27.056613283s) [noise=true , address=89.149.201.147:4024 , piece_id=7MCMBHNW4L7JDB4ATRHAM3GWVFAYH34LWFNX5O7C6EXIB34BNWRA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228954088 (189h52m27.056662491s) [seg_pos={0 0}, upload=197386]: Segment reached optimal threshold of 26 pieces. Dec 13 13:20:26.229843002 (189h52m27.057551404s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: read tcp 159.223.143.39:53228->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.229857857 (189h52m27.05756626s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230208353 (189h52m27.057916754s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7): protocol: read tcp 159.223.143.39:45318->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.230219436 (189h52m27.057927837s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230249796 (189h52m27.057958176s) [seg_pos={0 0}, upload=197386]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 13 13:20:26.230536147 (189h52m27.05824454s) [upload=197386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 13 13:20:26.230543508 (189h52m27.058251899s) [upload=197386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 13 13:20:26.379840216 (189h52m27.207548608s) [upload=197386]: Done uploading segments... Dec 13 13:20:26.379861196 (189h52m27.207569587s) [upload=197386]: Upload finished. err: <nil> Duration: 3.690893723s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 4.648285616s trace: https://jaeger.dev.storj.io/trace/12dbee1725a89af6 FAIL (2025-12-13 13:35:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=197386]: Uploading piece... Dec 13 13:20:22.157523997 (189h52m22.985232392s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157563526 (189h52m22.985271946s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157782226 (189h52m22.985490616s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158220136 (189h52m22.98592853s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158561593 (189h52m22.986269971s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158833133 (189h52m22.986541524s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159110305 (189h52m22.986818699s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159379461 (189h52m22.987087838s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159627778 (189h52m22.987336156s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.588165740 (189h52m23.41587413s) [upload=197386]: Next returned nil segment Dec 13 13:20:22.588188840 (189h52m23.415897231s) [upload=197386]: Waiting for error group managing segments... Dec 13 13:20:22.787355936 (189h52m23.615064332s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.793561088 (189h52m23.621269481s) [noise=true , address=185.167.204.205:4013 , piece_id=QNJLAFTMBQPQF7IBMT6EKMBEL5KX7VHU3L4L3ALPXYNUV2MZ43GA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.801170609 (189h52m23.628879002s) [noise=true , address=185.167.204.206:4068 , piece_id=P2FTDV3XW34Z6M73CWIURQCCQ6H5NXDNJWKBVN3GRQTDV65QLP5A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.812199494 (189h52m23.63990789s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813259194 (189h52m23.640967588s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813831001 (189h52m23.641539395s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814310957 (189h52m23.642019347s) [noise=true , address=185.167.204.205:4001 , piece_id=WXEZMLB4ZM2FFOB4JMCIF2X6FTB72I3EWWRTDLIZ6SXCMUTS5HGQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814961478 (189h52m23.64266987s) [noise=true , address=185.167.204.206:4046 , piece_id=KLHWCCQLK3CAOIMW57WAFZHNMECYLGA4VRZE3LXRUJRQCKAGGBKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.815469120 (189h52m23.643177512s) [noise=true , address=185.167.204.206:4054 , piece_id=ZPZSVEBAXMRI7732XKIHUGMU2WGGCNKE34U2FZ3NHJLKCMYBIUXQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.817780190 (189h52m23.645488584s) [noise=true , address=185.167.204.206:4045 , piece_id=VQ37X765RRBAVPI4HIAPJET7VRCI2FE7FRHB45II6ETFYDYMKSTQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818186706 (189h52m23.645895101s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818345893 (189h52m23.646054284s) [noise=true , address=185.167.204.205:4014 , piece_id=4U6VXOGIXKAV7NJBQBD2V6ZVWAUIT557OP7AUG332UWVS5LZFSPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.844792090 (189h52m23.672516847s) [noise=true , address=185.167.204.206:4051 , piece_id=A4VZAGPIVLUVOVOIGGIK7CEDRPUSV34XMUL5TFL5KFJNICAWSJ4Q, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.354540799 (189h52m25.182249185s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.369904036 (189h52m25.197612417s) [noise=true , address=81.171.14.70:4039 , piece_id=YV7FP35KY6TXYDKBEOV6RCHCSJWR4TQCFEB43GK3AZP4C776BCZQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.378758619 (189h52m25.206467012s) [noise=true , address=81.171.14.70:4047 , piece_id=EQFLTLGTZBKQ47MWFLSBCEHDKAWMZSGJHZPBAN67EANSP2LR7HYQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.384761037 (189h52m25.212469431s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.495107931 (189h52m25.322816323s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.833114410 (189h52m25.660822801s) [noise=true , address=89.149.201.147:4009 , piece_id=SAES2PXHWVMC22L3DKLBYHAI3ZHKYKGYOULL4ZZ2AB2U53B5URVA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.076006180 (189h52m25.903714572s) [noise=true , address=89.149.201.147:4008 , piece_id=HL2EBKXU5RWRNZP7D3MUOTRJK2YJTADFQS6C4IF7CGEUF3PLOIMQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.109256616 (189h52m25.936964999s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.121296573 (189h52m25.949004965s) [noise=true , address=81.171.14.70:4034 , piece_id=XZ72ODCL37AYLPMMFTCT4NJROVRCLTEM2XN6SQVG3G5NTZRJDWDA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.144861937 (189h52m25.972570318s) [noise=true , address=89.149.201.147:4012 , piece_id=5LDACT5ZGR54I4LNLZQHNXUY2VZQ5XIWI5AYUM5CZHCOOQIBGPRQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.859441967 (189h52m26.687150358s) [noise=true , address=81.171.14.70:4026 , piece_id=R7B2VE64HYLI7A2XVWDFTIIWAVLT6I6E75ZCAMTDH77R7BW2Q7IQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.932179545 (189h52m26.759887944s) [noise=true , address=89.149.201.147:4001 , piece_id=GDNVHYCJJDXLUDF77L6BP4IMIMR6KRZ2FTZJ6BG5TMAZGIW75PHQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228904886 (189h52m27.056613283s) [noise=true , address=89.149.201.147:4024 , piece_id=7MCMBHNW4L7JDB4ATRHAM3GWVFAYH34LWFNX5O7C6EXIB34BNWRA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228954088 (189h52m27.056662491s) [seg_pos={0 0}, upload=197386]: Segment reached optimal threshold of 26 pieces. Dec 13 13:20:26.229843002 (189h52m27.057551404s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: read tcp 159.223.143.39:53228->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.229857857 (189h52m27.05756626s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230208353 (189h52m27.057916754s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7): protocol: read tcp 159.223.143.39:45318->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.230219436 (189h52m27.057927837s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230249796 (189h52m27.057958176s) [seg_pos={0 0}, upload=197386]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 13 13:20:26.230536147 (189h52m27.05824454s) [upload=197386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 13 13:20:26.230543508 (189h52m27.058251899s) [upload=197386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 13 13:20:26.379840216 (189h52m27.207548608s) [upload=197386]: Done uploading segments... Dec 13 13:20:26.379861196 (189h52m27.207569587s) [upload=197386]: Upload finished. err: <nil> Duration: 4.648313402s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 3.930510836s trace: https://jaeger.dev.storj.io/trace/12dbee1725a89af6 FAIL (2025-12-13 13:35:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=197386]: Uploading piece... Dec 13 13:20:22.157523997 (189h52m22.985232392s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157563526 (189h52m22.985271946s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157782226 (189h52m22.985490616s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158220136 (189h52m22.98592853s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158561593 (189h52m22.986269971s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158833133 (189h52m22.986541524s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159110305 (189h52m22.986818699s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159379461 (189h52m22.987087838s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159627778 (189h52m22.987336156s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.588165740 (189h52m23.41587413s) [upload=197386]: Next returned nil segment Dec 13 13:20:22.588188840 (189h52m23.415897231s) [upload=197386]: Waiting for error group managing segments... Dec 13 13:20:22.787355936 (189h52m23.615064332s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.793561088 (189h52m23.621269481s) [noise=true , address=185.167.204.205:4013 , piece_id=QNJLAFTMBQPQF7IBMT6EKMBEL5KX7VHU3L4L3ALPXYNUV2MZ43GA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.801170609 (189h52m23.628879002s) [noise=true , address=185.167.204.206:4068 , piece_id=P2FTDV3XW34Z6M73CWIURQCCQ6H5NXDNJWKBVN3GRQTDV65QLP5A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.812199494 (189h52m23.63990789s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813259194 (189h52m23.640967588s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813831001 (189h52m23.641539395s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814310957 (189h52m23.642019347s) [noise=true , address=185.167.204.205:4001 , piece_id=WXEZMLB4ZM2FFOB4JMCIF2X6FTB72I3EWWRTDLIZ6SXCMUTS5HGQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814961478 (189h52m23.64266987s) [noise=true , address=185.167.204.206:4046 , piece_id=KLHWCCQLK3CAOIMW57WAFZHNMECYLGA4VRZE3LXRUJRQCKAGGBKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.815469120 (189h52m23.643177512s) [noise=true , address=185.167.204.206:4054 , piece_id=ZPZSVEBAXMRI7732XKIHUGMU2WGGCNKE34U2FZ3NHJLKCMYBIUXQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.817780190 (189h52m23.645488584s) [noise=true , address=185.167.204.206:4045 , piece_id=VQ37X765RRBAVPI4HIAPJET7VRCI2FE7FRHB45II6ETFYDYMKSTQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818186706 (189h52m23.645895101s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818345893 (189h52m23.646054284s) [noise=true , address=185.167.204.205:4014 , piece_id=4U6VXOGIXKAV7NJBQBD2V6ZVWAUIT557OP7AUG332UWVS5LZFSPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.844792090 (189h52m23.672516847s) [noise=true , address=185.167.204.206:4051 , piece_id=A4VZAGPIVLUVOVOIGGIK7CEDRPUSV34XMUL5TFL5KFJNICAWSJ4Q, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.354540799 (189h52m25.182249185s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.369904036 (189h52m25.197612417s) [noise=true , address=81.171.14.70:4039 , piece_id=YV7FP35KY6TXYDKBEOV6RCHCSJWR4TQCFEB43GK3AZP4C776BCZQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.378758619 (189h52m25.206467012s) [noise=true , address=81.171.14.70:4047 , piece_id=EQFLTLGTZBKQ47MWFLSBCEHDKAWMZSGJHZPBAN67EANSP2LR7HYQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.384761037 (189h52m25.212469431s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.495107931 (189h52m25.322816323s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.833114410 (189h52m25.660822801s) [noise=true , address=89.149.201.147:4009 , piece_id=SAES2PXHWVMC22L3DKLBYHAI3ZHKYKGYOULL4ZZ2AB2U53B5URVA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.076006180 (189h52m25.903714572s) [noise=true , address=89.149.201.147:4008 , piece_id=HL2EBKXU5RWRNZP7D3MUOTRJK2YJTADFQS6C4IF7CGEUF3PLOIMQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.109256616 (189h52m25.936964999s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.121296573 (189h52m25.949004965s) [noise=true , address=81.171.14.70:4034 , piece_id=XZ72ODCL37AYLPMMFTCT4NJROVRCLTEM2XN6SQVG3G5NTZRJDWDA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.144861937 (189h52m25.972570318s) [noise=true , address=89.149.201.147:4012 , piece_id=5LDACT5ZGR54I4LNLZQHNXUY2VZQ5XIWI5AYUM5CZHCOOQIBGPRQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.859441967 (189h52m26.687150358s) [noise=true , address=81.171.14.70:4026 , piece_id=R7B2VE64HYLI7A2XVWDFTIIWAVLT6I6E75ZCAMTDH77R7BW2Q7IQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.932179545 (189h52m26.759887944s) [noise=true , address=89.149.201.147:4001 , piece_id=GDNVHYCJJDXLUDF77L6BP4IMIMR6KRZ2FTZJ6BG5TMAZGIW75PHQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228904886 (189h52m27.056613283s) [noise=true , address=89.149.201.147:4024 , piece_id=7MCMBHNW4L7JDB4ATRHAM3GWVFAYH34LWFNX5O7C6EXIB34BNWRA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228954088 (189h52m27.056662491s) [seg_pos={0 0}, upload=197386]: Segment reached optimal threshold of 26 pieces. Dec 13 13:20:26.229843002 (189h52m27.057551404s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: read tcp 159.223.143.39:53228->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.229857857 (189h52m27.05756626s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230208353 (189h52m27.057916754s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7): protocol: read tcp 159.223.143.39:45318->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.230219436 (189h52m27.057927837s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230249796 (189h52m27.057958176s) [seg_pos={0 0}, upload=197386]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 13 13:20:26.230536147 (189h52m27.05824454s) [upload=197386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 13 13:20:26.230543508 (189h52m27.058251899s) [upload=197386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 13 13:20:26.379840216 (189h52m27.207548608s) [upload=197386]: Done uploading segments... Dec 13 13:20:26.379861196 (189h52m27.207569587s) [upload=197386]: Upload finished. err: <nil> Duration: 3.930548806s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 3.891010114s trace: https://jaeger.dev.storj.io/trace/12dbee1725a89af6 FAIL (2025-12-13 13:34:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=197386]: Uploading piece... Dec 13 13:20:22.157523997 (189h52m22.985232392s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157563526 (189h52m22.985271946s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157782226 (189h52m22.985490616s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158220136 (189h52m22.98592853s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158561593 (189h52m22.986269971s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158833133 (189h52m22.986541524s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159110305 (189h52m22.986818699s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159379461 (189h52m22.987087838s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159627778 (189h52m22.987336156s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.588165740 (189h52m23.41587413s) [upload=197386]: Next returned nil segment Dec 13 13:20:22.588188840 (189h52m23.415897231s) [upload=197386]: Waiting for error group managing segments... Dec 13 13:20:22.787355936 (189h52m23.615064332s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.793561088 (189h52m23.621269481s) [noise=true , address=185.167.204.205:4013 , piece_id=QNJLAFTMBQPQF7IBMT6EKMBEL5KX7VHU3L4L3ALPXYNUV2MZ43GA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.801170609 (189h52m23.628879002s) [noise=true , address=185.167.204.206:4068 , piece_id=P2FTDV3XW34Z6M73CWIURQCCQ6H5NXDNJWKBVN3GRQTDV65QLP5A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.812199494 (189h52m23.63990789s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813259194 (189h52m23.640967588s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813831001 (189h52m23.641539395s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814310957 (189h52m23.642019347s) [noise=true , address=185.167.204.205:4001 , piece_id=WXEZMLB4ZM2FFOB4JMCIF2X6FTB72I3EWWRTDLIZ6SXCMUTS5HGQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814961478 (189h52m23.64266987s) [noise=true , address=185.167.204.206:4046 , piece_id=KLHWCCQLK3CAOIMW57WAFZHNMECYLGA4VRZE3LXRUJRQCKAGGBKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.815469120 (189h52m23.643177512s) [noise=true , address=185.167.204.206:4054 , piece_id=ZPZSVEBAXMRI7732XKIHUGMU2WGGCNKE34U2FZ3NHJLKCMYBIUXQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.817780190 (189h52m23.645488584s) [noise=true , address=185.167.204.206:4045 , piece_id=VQ37X765RRBAVPI4HIAPJET7VRCI2FE7FRHB45II6ETFYDYMKSTQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818186706 (189h52m23.645895101s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818345893 (189h52m23.646054284s) [noise=true , address=185.167.204.205:4014 , piece_id=4U6VXOGIXKAV7NJBQBD2V6ZVWAUIT557OP7AUG332UWVS5LZFSPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.844792090 (189h52m23.672516847s) [noise=true , address=185.167.204.206:4051 , piece_id=A4VZAGPIVLUVOVOIGGIK7CEDRPUSV34XMUL5TFL5KFJNICAWSJ4Q, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.354540799 (189h52m25.182249185s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.369904036 (189h52m25.197612417s) [noise=true , address=81.171.14.70:4039 , piece_id=YV7FP35KY6TXYDKBEOV6RCHCSJWR4TQCFEB43GK3AZP4C776BCZQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.378758619 (189h52m25.206467012s) [noise=true , address=81.171.14.70:4047 , piece_id=EQFLTLGTZBKQ47MWFLSBCEHDKAWMZSGJHZPBAN67EANSP2LR7HYQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.384761037 (189h52m25.212469431s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.495107931 (189h52m25.322816323s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.833114410 (189h52m25.660822801s) [noise=true , address=89.149.201.147:4009 , piece_id=SAES2PXHWVMC22L3DKLBYHAI3ZHKYKGYOULL4ZZ2AB2U53B5URVA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.076006180 (189h52m25.903714572s) [noise=true , address=89.149.201.147:4008 , piece_id=HL2EBKXU5RWRNZP7D3MUOTRJK2YJTADFQS6C4IF7CGEUF3PLOIMQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.109256616 (189h52m25.936964999s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.121296573 (189h52m25.949004965s) [noise=true , address=81.171.14.70:4034 , piece_id=XZ72ODCL37AYLPMMFTCT4NJROVRCLTEM2XN6SQVG3G5NTZRJDWDA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.144861937 (189h52m25.972570318s) [noise=true , address=89.149.201.147:4012 , piece_id=5LDACT5ZGR54I4LNLZQHNXUY2VZQ5XIWI5AYUM5CZHCOOQIBGPRQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.859441967 (189h52m26.687150358s) [noise=true , address=81.171.14.70:4026 , piece_id=R7B2VE64HYLI7A2XVWDFTIIWAVLT6I6E75ZCAMTDH77R7BW2Q7IQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.932179545 (189h52m26.759887944s) [noise=true , address=89.149.201.147:4001 , piece_id=GDNVHYCJJDXLUDF77L6BP4IMIMR6KRZ2FTZJ6BG5TMAZGIW75PHQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228904886 (189h52m27.056613283s) [noise=true , address=89.149.201.147:4024 , piece_id=7MCMBHNW4L7JDB4ATRHAM3GWVFAYH34LWFNX5O7C6EXIB34BNWRA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228954088 (189h52m27.056662491s) [seg_pos={0 0}, upload=197386]: Segment reached optimal threshold of 26 pieces. Dec 13 13:20:26.229843002 (189h52m27.057551404s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: read tcp 159.223.143.39:53228->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.229857857 (189h52m27.05756626s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230208353 (189h52m27.057916754s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7): protocol: read tcp 159.223.143.39:45318->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.230219436 (189h52m27.057927837s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230249796 (189h52m27.057958176s) [seg_pos={0 0}, upload=197386]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 13 13:20:26.230536147 (189h52m27.05824454s) [upload=197386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 13 13:20:26.230543508 (189h52m27.058251899s) [upload=197386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 13 13:20:26.379840216 (189h52m27.207548608s) [upload=197386]: Done uploading segments... Dec 13 13:20:26.379861196 (189h52m27.207569587s) [upload=197386]: Upload finished. err: <nil> Duration: 3.891045808s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 3.68703121s trace: https://jaeger.dev.storj.io/trace/12dbee1725a89af6 FAIL (2025-12-13 13:33:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=197386]: Uploading piece... Dec 13 13:20:22.157523997 (189h52m22.985232392s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157563526 (189h52m22.985271946s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157782226 (189h52m22.985490616s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158220136 (189h52m22.98592853s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158561593 (189h52m22.986269971s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158833133 (189h52m22.986541524s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159110305 (189h52m22.986818699s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159379461 (189h52m22.987087838s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159627778 (189h52m22.987336156s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.588165740 (189h52m23.41587413s) [upload=197386]: Next returned nil segment Dec 13 13:20:22.588188840 (189h52m23.415897231s) [upload=197386]: Waiting for error group managing segments... Dec 13 13:20:22.787355936 (189h52m23.615064332s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.793561088 (189h52m23.621269481s) [noise=true , address=185.167.204.205:4013 , piece_id=QNJLAFTMBQPQF7IBMT6EKMBEL5KX7VHU3L4L3ALPXYNUV2MZ43GA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.801170609 (189h52m23.628879002s) [noise=true , address=185.167.204.206:4068 , piece_id=P2FTDV3XW34Z6M73CWIURQCCQ6H5NXDNJWKBVN3GRQTDV65QLP5A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.812199494 (189h52m23.63990789s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813259194 (189h52m23.640967588s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813831001 (189h52m23.641539395s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814310957 (189h52m23.642019347s) [noise=true , address=185.167.204.205:4001 , piece_id=WXEZMLB4ZM2FFOB4JMCIF2X6FTB72I3EWWRTDLIZ6SXCMUTS5HGQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814961478 (189h52m23.64266987s) [noise=true , address=185.167.204.206:4046 , piece_id=KLHWCCQLK3CAOIMW57WAFZHNMECYLGA4VRZE3LXRUJRQCKAGGBKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.815469120 (189h52m23.643177512s) [noise=true , address=185.167.204.206:4054 , piece_id=ZPZSVEBAXMRI7732XKIHUGMU2WGGCNKE34U2FZ3NHJLKCMYBIUXQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.817780190 (189h52m23.645488584s) [noise=true , address=185.167.204.206:4045 , piece_id=VQ37X765RRBAVPI4HIAPJET7VRCI2FE7FRHB45II6ETFYDYMKSTQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818186706 (189h52m23.645895101s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818345893 (189h52m23.646054284s) [noise=true , address=185.167.204.205:4014 , piece_id=4U6VXOGIXKAV7NJBQBD2V6ZVWAUIT557OP7AUG332UWVS5LZFSPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.844792090 (189h52m23.672516847s) [noise=true , address=185.167.204.206:4051 , piece_id=A4VZAGPIVLUVOVOIGGIK7CEDRPUSV34XMUL5TFL5KFJNICAWSJ4Q, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.354540799 (189h52m25.182249185s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.369904036 (189h52m25.197612417s) [noise=true , address=81.171.14.70:4039 , piece_id=YV7FP35KY6TXYDKBEOV6RCHCSJWR4TQCFEB43GK3AZP4C776BCZQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.378758619 (189h52m25.206467012s) [noise=true , address=81.171.14.70:4047 , piece_id=EQFLTLGTZBKQ47MWFLSBCEHDKAWMZSGJHZPBAN67EANSP2LR7HYQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.384761037 (189h52m25.212469431s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.495107931 (189h52m25.322816323s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.833114410 (189h52m25.660822801s) [noise=true , address=89.149.201.147:4009 , piece_id=SAES2PXHWVMC22L3DKLBYHAI3ZHKYKGYOULL4ZZ2AB2U53B5URVA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.076006180 (189h52m25.903714572s) [noise=true , address=89.149.201.147:4008 , piece_id=HL2EBKXU5RWRNZP7D3MUOTRJK2YJTADFQS6C4IF7CGEUF3PLOIMQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.109256616 (189h52m25.936964999s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.121296573 (189h52m25.949004965s) [noise=true , address=81.171.14.70:4034 , piece_id=XZ72ODCL37AYLPMMFTCT4NJROVRCLTEM2XN6SQVG3G5NTZRJDWDA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.144861937 (189h52m25.972570318s) [noise=true , address=89.149.201.147:4012 , piece_id=5LDACT5ZGR54I4LNLZQHNXUY2VZQ5XIWI5AYUM5CZHCOOQIBGPRQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.859441967 (189h52m26.687150358s) [noise=true , address=81.171.14.70:4026 , piece_id=R7B2VE64HYLI7A2XVWDFTIIWAVLT6I6E75ZCAMTDH77R7BW2Q7IQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.932179545 (189h52m26.759887944s) [noise=true , address=89.149.201.147:4001 , piece_id=GDNVHYCJJDXLUDF77L6BP4IMIMR6KRZ2FTZJ6BG5TMAZGIW75PHQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228904886 (189h52m27.056613283s) [noise=true , address=89.149.201.147:4024 , piece_id=7MCMBHNW4L7JDB4ATRHAM3GWVFAYH34LWFNX5O7C6EXIB34BNWRA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228954088 (189h52m27.056662491s) [seg_pos={0 0}, upload=197386]: Segment reached optimal threshold of 26 pieces. Dec 13 13:20:26.229843002 (189h52m27.057551404s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: read tcp 159.223.143.39:53228->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.229857857 (189h52m27.05756626s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230208353 (189h52m27.057916754s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7): protocol: read tcp 159.223.143.39:45318->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.230219436 (189h52m27.057927837s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230249796 (189h52m27.057958176s) [seg_pos={0 0}, upload=197386]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 13 13:20:26.230536147 (189h52m27.05824454s) [upload=197386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 13 13:20:26.230543508 (189h52m27.058251899s) [upload=197386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 13 13:20:26.379840216 (189h52m27.207548608s) [upload=197386]: Done uploading segments... Dec 13 13:20:26.379861196 (189h52m27.207569587s) [upload=197386]: Upload finished. err: <nil> Duration: 3.687063499s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.005050558s trace: https://jaeger.dev.storj.io/trace/12dbee1725a89af6 FAIL (2025-12-13 13:33:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=197386]: Uploading piece... Dec 13 13:20:22.157523997 (189h52m22.985232392s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157563526 (189h52m22.985271946s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157782226 (189h52m22.985490616s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158220136 (189h52m22.98592853s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158561593 (189h52m22.986269971s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158833133 (189h52m22.986541524s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159110305 (189h52m22.986818699s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159379461 (189h52m22.987087838s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159627778 (189h52m22.987336156s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.588165740 (189h52m23.41587413s) [upload=197386]: Next returned nil segment Dec 13 13:20:22.588188840 (189h52m23.415897231s) [upload=197386]: Waiting for error group managing segments... Dec 13 13:20:22.787355936 (189h52m23.615064332s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.793561088 (189h52m23.621269481s) [noise=true , address=185.167.204.205:4013 , piece_id=QNJLAFTMBQPQF7IBMT6EKMBEL5KX7VHU3L4L3ALPXYNUV2MZ43GA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.801170609 (189h52m23.628879002s) [noise=true , address=185.167.204.206:4068 , piece_id=P2FTDV3XW34Z6M73CWIURQCCQ6H5NXDNJWKBVN3GRQTDV65QLP5A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.812199494 (189h52m23.63990789s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813259194 (189h52m23.640967588s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813831001 (189h52m23.641539395s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814310957 (189h52m23.642019347s) [noise=true , address=185.167.204.205:4001 , piece_id=WXEZMLB4ZM2FFOB4JMCIF2X6FTB72I3EWWRTDLIZ6SXCMUTS5HGQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814961478 (189h52m23.64266987s) [noise=true , address=185.167.204.206:4046 , piece_id=KLHWCCQLK3CAOIMW57WAFZHNMECYLGA4VRZE3LXRUJRQCKAGGBKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.815469120 (189h52m23.643177512s) [noise=true , address=185.167.204.206:4054 , piece_id=ZPZSVEBAXMRI7732XKIHUGMU2WGGCNKE34U2FZ3NHJLKCMYBIUXQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.817780190 (189h52m23.645488584s) [noise=true , address=185.167.204.206:4045 , piece_id=VQ37X765RRBAVPI4HIAPJET7VRCI2FE7FRHB45II6ETFYDYMKSTQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818186706 (189h52m23.645895101s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818345893 (189h52m23.646054284s) [noise=true , address=185.167.204.205:4014 , piece_id=4U6VXOGIXKAV7NJBQBD2V6ZVWAUIT557OP7AUG332UWVS5LZFSPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.844792090 (189h52m23.672516847s) [noise=true , address=185.167.204.206:4051 , piece_id=A4VZAGPIVLUVOVOIGGIK7CEDRPUSV34XMUL5TFL5KFJNICAWSJ4Q, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.354540799 (189h52m25.182249185s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.369904036 (189h52m25.197612417s) [noise=true , address=81.171.14.70:4039 , piece_id=YV7FP35KY6TXYDKBEOV6RCHCSJWR4TQCFEB43GK3AZP4C776BCZQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.378758619 (189h52m25.206467012s) [noise=true , address=81.171.14.70:4047 , piece_id=EQFLTLGTZBKQ47MWFLSBCEHDKAWMZSGJHZPBAN67EANSP2LR7HYQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.384761037 (189h52m25.212469431s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.495107931 (189h52m25.322816323s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.833114410 (189h52m25.660822801s) [noise=true , address=89.149.201.147:4009 , piece_id=SAES2PXHWVMC22L3DKLBYHAI3ZHKYKGYOULL4ZZ2AB2U53B5URVA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.076006180 (189h52m25.903714572s) [noise=true , address=89.149.201.147:4008 , piece_id=HL2EBKXU5RWRNZP7D3MUOTRJK2YJTADFQS6C4IF7CGEUF3PLOIMQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.109256616 (189h52m25.936964999s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.121296573 (189h52m25.949004965s) [noise=true , address=81.171.14.70:4034 , piece_id=XZ72ODCL37AYLPMMFTCT4NJROVRCLTEM2XN6SQVG3G5NTZRJDWDA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.144861937 (189h52m25.972570318s) [noise=true , address=89.149.201.147:4012 , piece_id=5LDACT5ZGR54I4LNLZQHNXUY2VZQ5XIWI5AYUM5CZHCOOQIBGPRQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.859441967 (189h52m26.687150358s) [noise=true , address=81.171.14.70:4026 , piece_id=R7B2VE64HYLI7A2XVWDFTIIWAVLT6I6E75ZCAMTDH77R7BW2Q7IQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.932179545 (189h52m26.759887944s) [noise=true , address=89.149.201.147:4001 , piece_id=GDNVHYCJJDXLUDF77L6BP4IMIMR6KRZ2FTZJ6BG5TMAZGIW75PHQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228904886 (189h52m27.056613283s) [noise=true , address=89.149.201.147:4024 , piece_id=7MCMBHNW4L7JDB4ATRHAM3GWVFAYH34LWFNX5O7C6EXIB34BNWRA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228954088 (189h52m27.056662491s) [seg_pos={0 0}, upload=197386]: Segment reached optimal threshold of 26 pieces. Dec 13 13:20:26.229843002 (189h52m27.057551404s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: read tcp 159.223.143.39:53228->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.229857857 (189h52m27.05756626s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230208353 (189h52m27.057916754s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7): protocol: read tcp 159.223.143.39:45318->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.230219436 (189h52m27.057927837s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230249796 (189h52m27.057958176s) [seg_pos={0 0}, upload=197386]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 13 13:20:26.230536147 (189h52m27.05824454s) [upload=197386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 13 13:20:26.230543508 (189h52m27.058251899s) [upload=197386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 13 13:20:26.379840216 (189h52m27.207548608s) [upload=197386]: Done uploading segments... Dec 13 13:20:26.379861196 (189h52m27.207569587s) [upload=197386]: Upload finished. err: <nil> Duration: 5.005081974s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 8.52141402s trace: https://jaeger.dev.storj.io/trace/12dbee1725a89af6 FAIL (2025-12-13 13:32:44 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=197386]: Uploading piece... Dec 13 13:20:22.157523997 (189h52m22.985232392s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157563526 (189h52m22.985271946s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157782226 (189h52m22.985490616s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158220136 (189h52m22.98592853s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158561593 (189h52m22.986269971s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158833133 (189h52m22.986541524s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159110305 (189h52m22.986818699s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159379461 (189h52m22.987087838s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159627778 (189h52m22.987336156s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.588165740 (189h52m23.41587413s) [upload=197386]: Next returned nil segment Dec 13 13:20:22.588188840 (189h52m23.415897231s) [upload=197386]: Waiting for error group managing segments... Dec 13 13:20:22.787355936 (189h52m23.615064332s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.793561088 (189h52m23.621269481s) [noise=true , address=185.167.204.205:4013 , piece_id=QNJLAFTMBQPQF7IBMT6EKMBEL5KX7VHU3L4L3ALPXYNUV2MZ43GA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.801170609 (189h52m23.628879002s) [noise=true , address=185.167.204.206:4068 , piece_id=P2FTDV3XW34Z6M73CWIURQCCQ6H5NXDNJWKBVN3GRQTDV65QLP5A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.812199494 (189h52m23.63990789s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813259194 (189h52m23.640967588s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813831001 (189h52m23.641539395s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814310957 (189h52m23.642019347s) [noise=true , address=185.167.204.205:4001 , piece_id=WXEZMLB4ZM2FFOB4JMCIF2X6FTB72I3EWWRTDLIZ6SXCMUTS5HGQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814961478 (189h52m23.64266987s) [noise=true , address=185.167.204.206:4046 , piece_id=KLHWCCQLK3CAOIMW57WAFZHNMECYLGA4VRZE3LXRUJRQCKAGGBKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.815469120 (189h52m23.643177512s) [noise=true , address=185.167.204.206:4054 , piece_id=ZPZSVEBAXMRI7732XKIHUGMU2WGGCNKE34U2FZ3NHJLKCMYBIUXQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.817780190 (189h52m23.645488584s) [noise=true , address=185.167.204.206:4045 , piece_id=VQ37X765RRBAVPI4HIAPJET7VRCI2FE7FRHB45II6ETFYDYMKSTQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818186706 (189h52m23.645895101s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818345893 (189h52m23.646054284s) [noise=true , address=185.167.204.205:4014 , piece_id=4U6VXOGIXKAV7NJBQBD2V6ZVWAUIT557OP7AUG332UWVS5LZFSPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.844792090 (189h52m23.672516847s) [noise=true , address=185.167.204.206:4051 , piece_id=A4VZAGPIVLUVOVOIGGIK7CEDRPUSV34XMUL5TFL5KFJNICAWSJ4Q, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.354540799 (189h52m25.182249185s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.369904036 (189h52m25.197612417s) [noise=true , address=81.171.14.70:4039 , piece_id=YV7FP35KY6TXYDKBEOV6RCHCSJWR4TQCFEB43GK3AZP4C776BCZQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.378758619 (189h52m25.206467012s) [noise=true , address=81.171.14.70:4047 , piece_id=EQFLTLGTZBKQ47MWFLSBCEHDKAWMZSGJHZPBAN67EANSP2LR7HYQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.384761037 (189h52m25.212469431s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.495107931 (189h52m25.322816323s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.833114410 (189h52m25.660822801s) [noise=true , address=89.149.201.147:4009 , piece_id=SAES2PXHWVMC22L3DKLBYHAI3ZHKYKGYOULL4ZZ2AB2U53B5URVA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.076006180 (189h52m25.903714572s) [noise=true , address=89.149.201.147:4008 , piece_id=HL2EBKXU5RWRNZP7D3MUOTRJK2YJTADFQS6C4IF7CGEUF3PLOIMQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.109256616 (189h52m25.936964999s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.121296573 (189h52m25.949004965s) [noise=true , address=81.171.14.70:4034 , piece_id=XZ72ODCL37AYLPMMFTCT4NJROVRCLTEM2XN6SQVG3G5NTZRJDWDA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.144861937 (189h52m25.972570318s) [noise=true , address=89.149.201.147:4012 , piece_id=5LDACT5ZGR54I4LNLZQHNXUY2VZQ5XIWI5AYUM5CZHCOOQIBGPRQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.859441967 (189h52m26.687150358s) [noise=true , address=81.171.14.70:4026 , piece_id=R7B2VE64HYLI7A2XVWDFTIIWAVLT6I6E75ZCAMTDH77R7BW2Q7IQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.932179545 (189h52m26.759887944s) [noise=true , address=89.149.201.147:4001 , piece_id=GDNVHYCJJDXLUDF77L6BP4IMIMR6KRZ2FTZJ6BG5TMAZGIW75PHQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228904886 (189h52m27.056613283s) [noise=true , address=89.149.201.147:4024 , piece_id=7MCMBHNW4L7JDB4ATRHAM3GWVFAYH34LWFNX5O7C6EXIB34BNWRA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228954088 (189h52m27.056662491s) [seg_pos={0 0}, upload=197386]: Segment reached optimal threshold of 26 pieces. Dec 13 13:20:26.229843002 (189h52m27.057551404s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: read tcp 159.223.143.39:53228->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.229857857 (189h52m27.05756626s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230208353 (189h52m27.057916754s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7): protocol: read tcp 159.223.143.39:45318->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.230219436 (189h52m27.057927837s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230249796 (189h52m27.057958176s) [seg_pos={0 0}, upload=197386]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 13 13:20:26.230536147 (189h52m27.05824454s) [upload=197386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 13 13:20:26.230543508 (189h52m27.058251899s) [upload=197386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 13 13:20:26.379840216 (189h52m27.207548608s) [upload=197386]: Done uploading segments... Dec 13 13:20:26.379861196 (189h52m27.207569587s) [upload=197386]: Upload finished. err: <nil> Duration: 8.521442368s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 3.734737354s trace: https://jaeger.dev.storj.io/trace/12dbee1725a89af6 FAIL (2025-12-13 13:32:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=197386]: Uploading piece... Dec 13 13:20:22.157523997 (189h52m22.985232392s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157563526 (189h52m22.985271946s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157782226 (189h52m22.985490616s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158220136 (189h52m22.98592853s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158561593 (189h52m22.986269971s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158833133 (189h52m22.986541524s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159110305 (189h52m22.986818699s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159379461 (189h52m22.987087838s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159627778 (189h52m22.987336156s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.588165740 (189h52m23.41587413s) [upload=197386]: Next returned nil segment Dec 13 13:20:22.588188840 (189h52m23.415897231s) [upload=197386]: Waiting for error group managing segments... Dec 13 13:20:22.787355936 (189h52m23.615064332s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.793561088 (189h52m23.621269481s) [noise=true , address=185.167.204.205:4013 , piece_id=QNJLAFTMBQPQF7IBMT6EKMBEL5KX7VHU3L4L3ALPXYNUV2MZ43GA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.801170609 (189h52m23.628879002s) [noise=true , address=185.167.204.206:4068 , piece_id=P2FTDV3XW34Z6M73CWIURQCCQ6H5NXDNJWKBVN3GRQTDV65QLP5A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.812199494 (189h52m23.63990789s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813259194 (189h52m23.640967588s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813831001 (189h52m23.641539395s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814310957 (189h52m23.642019347s) [noise=true , address=185.167.204.205:4001 , piece_id=WXEZMLB4ZM2FFOB4JMCIF2X6FTB72I3EWWRTDLIZ6SXCMUTS5HGQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814961478 (189h52m23.64266987s) [noise=true , address=185.167.204.206:4046 , piece_id=KLHWCCQLK3CAOIMW57WAFZHNMECYLGA4VRZE3LXRUJRQCKAGGBKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.815469120 (189h52m23.643177512s) [noise=true , address=185.167.204.206:4054 , piece_id=ZPZSVEBAXMRI7732XKIHUGMU2WGGCNKE34U2FZ3NHJLKCMYBIUXQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.817780190 (189h52m23.645488584s) [noise=true , address=185.167.204.206:4045 , piece_id=VQ37X765RRBAVPI4HIAPJET7VRCI2FE7FRHB45II6ETFYDYMKSTQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818186706 (189h52m23.645895101s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818345893 (189h52m23.646054284s) [noise=true , address=185.167.204.205:4014 , piece_id=4U6VXOGIXKAV7NJBQBD2V6ZVWAUIT557OP7AUG332UWVS5LZFSPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.844792090 (189h52m23.672516847s) [noise=true , address=185.167.204.206:4051 , piece_id=A4VZAGPIVLUVOVOIGGIK7CEDRPUSV34XMUL5TFL5KFJNICAWSJ4Q, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.354540799 (189h52m25.182249185s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.369904036 (189h52m25.197612417s) [noise=true , address=81.171.14.70:4039 , piece_id=YV7FP35KY6TXYDKBEOV6RCHCSJWR4TQCFEB43GK3AZP4C776BCZQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.378758619 (189h52m25.206467012s) [noise=true , address=81.171.14.70:4047 , piece_id=EQFLTLGTZBKQ47MWFLSBCEHDKAWMZSGJHZPBAN67EANSP2LR7HYQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.384761037 (189h52m25.212469431s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.495107931 (189h52m25.322816323s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.833114410 (189h52m25.660822801s) [noise=true , address=89.149.201.147:4009 , piece_id=SAES2PXHWVMC22L3DKLBYHAI3ZHKYKGYOULL4ZZ2AB2U53B5URVA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.076006180 (189h52m25.903714572s) [noise=true , address=89.149.201.147:4008 , piece_id=HL2EBKXU5RWRNZP7D3MUOTRJK2YJTADFQS6C4IF7CGEUF3PLOIMQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.109256616 (189h52m25.936964999s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.121296573 (189h52m25.949004965s) [noise=true , address=81.171.14.70:4034 , piece_id=XZ72ODCL37AYLPMMFTCT4NJROVRCLTEM2XN6SQVG3G5NTZRJDWDA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.144861937 (189h52m25.972570318s) [noise=true , address=89.149.201.147:4012 , piece_id=5LDACT5ZGR54I4LNLZQHNXUY2VZQ5XIWI5AYUM5CZHCOOQIBGPRQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.859441967 (189h52m26.687150358s) [noise=true , address=81.171.14.70:4026 , piece_id=R7B2VE64HYLI7A2XVWDFTIIWAVLT6I6E75ZCAMTDH77R7BW2Q7IQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.932179545 (189h52m26.759887944s) [noise=true , address=89.149.201.147:4001 , piece_id=GDNVHYCJJDXLUDF77L6BP4IMIMR6KRZ2FTZJ6BG5TMAZGIW75PHQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228904886 (189h52m27.056613283s) [noise=true , address=89.149.201.147:4024 , piece_id=7MCMBHNW4L7JDB4ATRHAM3GWVFAYH34LWFNX5O7C6EXIB34BNWRA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228954088 (189h52m27.056662491s) [seg_pos={0 0}, upload=197386]: Segment reached optimal threshold of 26 pieces. Dec 13 13:20:26.229843002 (189h52m27.057551404s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: read tcp 159.223.143.39:53228->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.229857857 (189h52m27.05756626s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230208353 (189h52m27.057916754s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7): protocol: read tcp 159.223.143.39:45318->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.230219436 (189h52m27.057927837s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230249796 (189h52m27.057958176s) [seg_pos={0 0}, upload=197386]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 13 13:20:26.230536147 (189h52m27.05824454s) [upload=197386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 13 13:20:26.230543508 (189h52m27.058251899s) [upload=197386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 13 13:20:26.379840216 (189h52m27.207548608s) [upload=197386]: Done uploading segments... Dec 13 13:20:26.379861196 (189h52m27.207569587s) [upload=197386]: Upload finished. err: <nil> Duration: 3.734767926s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 6.367747587s trace: https://jaeger.dev.storj.io/trace/12dbee1725a89af6 FAIL (2025-12-13 13:31:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=197386]: Uploading piece... Dec 13 13:20:22.157523997 (189h52m22.985232392s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157563526 (189h52m22.985271946s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157782226 (189h52m22.985490616s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158220136 (189h52m22.98592853s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158561593 (189h52m22.986269971s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158833133 (189h52m22.986541524s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159110305 (189h52m22.986818699s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159379461 (189h52m22.987087838s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159627778 (189h52m22.987336156s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.588165740 (189h52m23.41587413s) [upload=197386]: Next returned nil segment Dec 13 13:20:22.588188840 (189h52m23.415897231s) [upload=197386]: Waiting for error group managing segments... Dec 13 13:20:22.787355936 (189h52m23.615064332s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.793561088 (189h52m23.621269481s) [noise=true , address=185.167.204.205:4013 , piece_id=QNJLAFTMBQPQF7IBMT6EKMBEL5KX7VHU3L4L3ALPXYNUV2MZ43GA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.801170609 (189h52m23.628879002s) [noise=true , address=185.167.204.206:4068 , piece_id=P2FTDV3XW34Z6M73CWIURQCCQ6H5NXDNJWKBVN3GRQTDV65QLP5A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.812199494 (189h52m23.63990789s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813259194 (189h52m23.640967588s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813831001 (189h52m23.641539395s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814310957 (189h52m23.642019347s) [noise=true , address=185.167.204.205:4001 , piece_id=WXEZMLB4ZM2FFOB4JMCIF2X6FTB72I3EWWRTDLIZ6SXCMUTS5HGQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814961478 (189h52m23.64266987s) [noise=true , address=185.167.204.206:4046 , piece_id=KLHWCCQLK3CAOIMW57WAFZHNMECYLGA4VRZE3LXRUJRQCKAGGBKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.815469120 (189h52m23.643177512s) [noise=true , address=185.167.204.206:4054 , piece_id=ZPZSVEBAXMRI7732XKIHUGMU2WGGCNKE34U2FZ3NHJLKCMYBIUXQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.817780190 (189h52m23.645488584s) [noise=true , address=185.167.204.206:4045 , piece_id=VQ37X765RRBAVPI4HIAPJET7VRCI2FE7FRHB45II6ETFYDYMKSTQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818186706 (189h52m23.645895101s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818345893 (189h52m23.646054284s) [noise=true , address=185.167.204.205:4014 , piece_id=4U6VXOGIXKAV7NJBQBD2V6ZVWAUIT557OP7AUG332UWVS5LZFSPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.844792090 (189h52m23.672516847s) [noise=true , address=185.167.204.206:4051 , piece_id=A4VZAGPIVLUVOVOIGGIK7CEDRPUSV34XMUL5TFL5KFJNICAWSJ4Q, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.354540799 (189h52m25.182249185s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.369904036 (189h52m25.197612417s) [noise=true , address=81.171.14.70:4039 , piece_id=YV7FP35KY6TXYDKBEOV6RCHCSJWR4TQCFEB43GK3AZP4C776BCZQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.378758619 (189h52m25.206467012s) [noise=true , address=81.171.14.70:4047 , piece_id=EQFLTLGTZBKQ47MWFLSBCEHDKAWMZSGJHZPBAN67EANSP2LR7HYQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.384761037 (189h52m25.212469431s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.495107931 (189h52m25.322816323s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.833114410 (189h52m25.660822801s) [noise=true , address=89.149.201.147:4009 , piece_id=SAES2PXHWVMC22L3DKLBYHAI3ZHKYKGYOULL4ZZ2AB2U53B5URVA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.076006180 (189h52m25.903714572s) [noise=true , address=89.149.201.147:4008 , piece_id=HL2EBKXU5RWRNZP7D3MUOTRJK2YJTADFQS6C4IF7CGEUF3PLOIMQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.109256616 (189h52m25.936964999s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.121296573 (189h52m25.949004965s) [noise=true , address=81.171.14.70:4034 , piece_id=XZ72ODCL37AYLPMMFTCT4NJROVRCLTEM2XN6SQVG3G5NTZRJDWDA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.144861937 (189h52m25.972570318s) [noise=true , address=89.149.201.147:4012 , piece_id=5LDACT5ZGR54I4LNLZQHNXUY2VZQ5XIWI5AYUM5CZHCOOQIBGPRQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.859441967 (189h52m26.687150358s) [noise=true , address=81.171.14.70:4026 , piece_id=R7B2VE64HYLI7A2XVWDFTIIWAVLT6I6E75ZCAMTDH77R7BW2Q7IQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.932179545 (189h52m26.759887944s) [noise=true , address=89.149.201.147:4001 , piece_id=GDNVHYCJJDXLUDF77L6BP4IMIMR6KRZ2FTZJ6BG5TMAZGIW75PHQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228904886 (189h52m27.056613283s) [noise=true , address=89.149.201.147:4024 , piece_id=7MCMBHNW4L7JDB4ATRHAM3GWVFAYH34LWFNX5O7C6EXIB34BNWRA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228954088 (189h52m27.056662491s) [seg_pos={0 0}, upload=197386]: Segment reached optimal threshold of 26 pieces. Dec 13 13:20:26.229843002 (189h52m27.057551404s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: read tcp 159.223.143.39:53228->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.229857857 (189h52m27.05756626s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230208353 (189h52m27.057916754s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7): protocol: read tcp 159.223.143.39:45318->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.230219436 (189h52m27.057927837s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230249796 (189h52m27.057958176s) [seg_pos={0 0}, upload=197386]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 13 13:20:26.230536147 (189h52m27.05824454s) [upload=197386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 13 13:20:26.230543508 (189h52m27.058251899s) [upload=197386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 13 13:20:26.379840216 (189h52m27.207548608s) [upload=197386]: Done uploading segments... Dec 13 13:20:26.379861196 (189h52m27.207569587s) [upload=197386]: Upload finished. err: <nil> Duration: 6.367777247s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 3.612020296s trace: https://jaeger.dev.storj.io/trace/12dbee1725a89af6 FAIL (2025-12-13 13:30:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=197386]: Uploading piece... Dec 13 13:20:22.157523997 (189h52m22.985232392s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157563526 (189h52m22.985271946s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157782226 (189h52m22.985490616s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158220136 (189h52m22.98592853s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158561593 (189h52m22.986269971s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158833133 (189h52m22.986541524s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159110305 (189h52m22.986818699s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159379461 (189h52m22.987087838s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159627778 (189h52m22.987336156s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.588165740 (189h52m23.41587413s) [upload=197386]: Next returned nil segment Dec 13 13:20:22.588188840 (189h52m23.415897231s) [upload=197386]: Waiting for error group managing segments... Dec 13 13:20:22.787355936 (189h52m23.615064332s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.793561088 (189h52m23.621269481s) [noise=true , address=185.167.204.205:4013 , piece_id=QNJLAFTMBQPQF7IBMT6EKMBEL5KX7VHU3L4L3ALPXYNUV2MZ43GA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.801170609 (189h52m23.628879002s) [noise=true , address=185.167.204.206:4068 , piece_id=P2FTDV3XW34Z6M73CWIURQCCQ6H5NXDNJWKBVN3GRQTDV65QLP5A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.812199494 (189h52m23.63990789s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813259194 (189h52m23.640967588s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813831001 (189h52m23.641539395s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814310957 (189h52m23.642019347s) [noise=true , address=185.167.204.205:4001 , piece_id=WXEZMLB4ZM2FFOB4JMCIF2X6FTB72I3EWWRTDLIZ6SXCMUTS5HGQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814961478 (189h52m23.64266987s) [noise=true , address=185.167.204.206:4046 , piece_id=KLHWCCQLK3CAOIMW57WAFZHNMECYLGA4VRZE3LXRUJRQCKAGGBKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.815469120 (189h52m23.643177512s) [noise=true , address=185.167.204.206:4054 , piece_id=ZPZSVEBAXMRI7732XKIHUGMU2WGGCNKE34U2FZ3NHJLKCMYBIUXQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.817780190 (189h52m23.645488584s) [noise=true , address=185.167.204.206:4045 , piece_id=VQ37X765RRBAVPI4HIAPJET7VRCI2FE7FRHB45II6ETFYDYMKSTQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818186706 (189h52m23.645895101s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818345893 (189h52m23.646054284s) [noise=true , address=185.167.204.205:4014 , piece_id=4U6VXOGIXKAV7NJBQBD2V6ZVWAUIT557OP7AUG332UWVS5LZFSPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.844792090 (189h52m23.672516847s) [noise=true , address=185.167.204.206:4051 , piece_id=A4VZAGPIVLUVOVOIGGIK7CEDRPUSV34XMUL5TFL5KFJNICAWSJ4Q, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.354540799 (189h52m25.182249185s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.369904036 (189h52m25.197612417s) [noise=true , address=81.171.14.70:4039 , piece_id=YV7FP35KY6TXYDKBEOV6RCHCSJWR4TQCFEB43GK3AZP4C776BCZQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.378758619 (189h52m25.206467012s) [noise=true , address=81.171.14.70:4047 , piece_id=EQFLTLGTZBKQ47MWFLSBCEHDKAWMZSGJHZPBAN67EANSP2LR7HYQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.384761037 (189h52m25.212469431s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.495107931 (189h52m25.322816323s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.833114410 (189h52m25.660822801s) [noise=true , address=89.149.201.147:4009 , piece_id=SAES2PXHWVMC22L3DKLBYHAI3ZHKYKGYOULL4ZZ2AB2U53B5URVA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.076006180 (189h52m25.903714572s) [noise=true , address=89.149.201.147:4008 , piece_id=HL2EBKXU5RWRNZP7D3MUOTRJK2YJTADFQS6C4IF7CGEUF3PLOIMQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.109256616 (189h52m25.936964999s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.121296573 (189h52m25.949004965s) [noise=true , address=81.171.14.70:4034 , piece_id=XZ72ODCL37AYLPMMFTCT4NJROVRCLTEM2XN6SQVG3G5NTZRJDWDA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.144861937 (189h52m25.972570318s) [noise=true , address=89.149.201.147:4012 , piece_id=5LDACT5ZGR54I4LNLZQHNXUY2VZQ5XIWI5AYUM5CZHCOOQIBGPRQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.859441967 (189h52m26.687150358s) [noise=true , address=81.171.14.70:4026 , piece_id=R7B2VE64HYLI7A2XVWDFTIIWAVLT6I6E75ZCAMTDH77R7BW2Q7IQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.932179545 (189h52m26.759887944s) [noise=true , address=89.149.201.147:4001 , piece_id=GDNVHYCJJDXLUDF77L6BP4IMIMR6KRZ2FTZJ6BG5TMAZGIW75PHQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228904886 (189h52m27.056613283s) [noise=true , address=89.149.201.147:4024 , piece_id=7MCMBHNW4L7JDB4ATRHAM3GWVFAYH34LWFNX5O7C6EXIB34BNWRA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228954088 (189h52m27.056662491s) [seg_pos={0 0}, upload=197386]: Segment reached optimal threshold of 26 pieces. Dec 13 13:20:26.229843002 (189h52m27.057551404s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: read tcp 159.223.143.39:53228->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.229857857 (189h52m27.05756626s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230208353 (189h52m27.057916754s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7): protocol: read tcp 159.223.143.39:45318->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.230219436 (189h52m27.057927837s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230249796 (189h52m27.057958176s) [seg_pos={0 0}, upload=197386]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 13 13:20:26.230536147 (189h52m27.05824454s) [upload=197386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 13 13:20:26.230543508 (189h52m27.058251899s) [upload=197386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 13 13:20:26.379840216 (189h52m27.207548608s) [upload=197386]: Done uploading segments... Dec 13 13:20:26.379861196 (189h52m27.207569587s) [upload=197386]: Upload finished. err: <nil> Duration: 3.612052032s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 3.610817965s trace: https://jaeger.dev.storj.io/trace/12dbee1725a89af6 FAIL (2025-12-13 13:30:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=197386]: Uploading piece... Dec 13 13:20:22.157523997 (189h52m22.985232392s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157563526 (189h52m22.985271946s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157782226 (189h52m22.985490616s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158220136 (189h52m22.98592853s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158561593 (189h52m22.986269971s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158833133 (189h52m22.986541524s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159110305 (189h52m22.986818699s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159379461 (189h52m22.987087838s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159627778 (189h52m22.987336156s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.588165740 (189h52m23.41587413s) [upload=197386]: Next returned nil segment Dec 13 13:20:22.588188840 (189h52m23.415897231s) [upload=197386]: Waiting for error group managing segments... Dec 13 13:20:22.787355936 (189h52m23.615064332s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.793561088 (189h52m23.621269481s) [noise=true , address=185.167.204.205:4013 , piece_id=QNJLAFTMBQPQF7IBMT6EKMBEL5KX7VHU3L4L3ALPXYNUV2MZ43GA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.801170609 (189h52m23.628879002s) [noise=true , address=185.167.204.206:4068 , piece_id=P2FTDV3XW34Z6M73CWIURQCCQ6H5NXDNJWKBVN3GRQTDV65QLP5A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.812199494 (189h52m23.63990789s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813259194 (189h52m23.640967588s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813831001 (189h52m23.641539395s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814310957 (189h52m23.642019347s) [noise=true , address=185.167.204.205:4001 , piece_id=WXEZMLB4ZM2FFOB4JMCIF2X6FTB72I3EWWRTDLIZ6SXCMUTS5HGQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814961478 (189h52m23.64266987s) [noise=true , address=185.167.204.206:4046 , piece_id=KLHWCCQLK3CAOIMW57WAFZHNMECYLGA4VRZE3LXRUJRQCKAGGBKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.815469120 (189h52m23.643177512s) [noise=true , address=185.167.204.206:4054 , piece_id=ZPZSVEBAXMRI7732XKIHUGMU2WGGCNKE34U2FZ3NHJLKCMYBIUXQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.817780190 (189h52m23.645488584s) [noise=true , address=185.167.204.206:4045 , piece_id=VQ37X765RRBAVPI4HIAPJET7VRCI2FE7FRHB45II6ETFYDYMKSTQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818186706 (189h52m23.645895101s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818345893 (189h52m23.646054284s) [noise=true , address=185.167.204.205:4014 , piece_id=4U6VXOGIXKAV7NJBQBD2V6ZVWAUIT557OP7AUG332UWVS5LZFSPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.844792090 (189h52m23.672516847s) [noise=true , address=185.167.204.206:4051 , piece_id=A4VZAGPIVLUVOVOIGGIK7CEDRPUSV34XMUL5TFL5KFJNICAWSJ4Q, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.354540799 (189h52m25.182249185s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.369904036 (189h52m25.197612417s) [noise=true , address=81.171.14.70:4039 , piece_id=YV7FP35KY6TXYDKBEOV6RCHCSJWR4TQCFEB43GK3AZP4C776BCZQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.378758619 (189h52m25.206467012s) [noise=true , address=81.171.14.70:4047 , piece_id=EQFLTLGTZBKQ47MWFLSBCEHDKAWMZSGJHZPBAN67EANSP2LR7HYQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.384761037 (189h52m25.212469431s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.495107931 (189h52m25.322816323s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.833114410 (189h52m25.660822801s) [noise=true , address=89.149.201.147:4009 , piece_id=SAES2PXHWVMC22L3DKLBYHAI3ZHKYKGYOULL4ZZ2AB2U53B5URVA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.076006180 (189h52m25.903714572s) [noise=true , address=89.149.201.147:4008 , piece_id=HL2EBKXU5RWRNZP7D3MUOTRJK2YJTADFQS6C4IF7CGEUF3PLOIMQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.109256616 (189h52m25.936964999s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.121296573 (189h52m25.949004965s) [noise=true , address=81.171.14.70:4034 , piece_id=XZ72ODCL37AYLPMMFTCT4NJROVRCLTEM2XN6SQVG3G5NTZRJDWDA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.144861937 (189h52m25.972570318s) [noise=true , address=89.149.201.147:4012 , piece_id=5LDACT5ZGR54I4LNLZQHNXUY2VZQ5XIWI5AYUM5CZHCOOQIBGPRQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.859441967 (189h52m26.687150358s) [noise=true , address=81.171.14.70:4026 , piece_id=R7B2VE64HYLI7A2XVWDFTIIWAVLT6I6E75ZCAMTDH77R7BW2Q7IQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.932179545 (189h52m26.759887944s) [noise=true , address=89.149.201.147:4001 , piece_id=GDNVHYCJJDXLUDF77L6BP4IMIMR6KRZ2FTZJ6BG5TMAZGIW75PHQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228904886 (189h52m27.056613283s) [noise=true , address=89.149.201.147:4024 , piece_id=7MCMBHNW4L7JDB4ATRHAM3GWVFAYH34LWFNX5O7C6EXIB34BNWRA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228954088 (189h52m27.056662491s) [seg_pos={0 0}, upload=197386]: Segment reached optimal threshold of 26 pieces. Dec 13 13:20:26.229843002 (189h52m27.057551404s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: read tcp 159.223.143.39:53228->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.229857857 (189h52m27.05756626s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230208353 (189h52m27.057916754s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7): protocol: read tcp 159.223.143.39:45318->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.230219436 (189h52m27.057927837s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230249796 (189h52m27.057958176s) [seg_pos={0 0}, upload=197386]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 13 13:20:26.230536147 (189h52m27.05824454s) [upload=197386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 13 13:20:26.230543508 (189h52m27.058251899s) [upload=197386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 13 13:20:26.379840216 (189h52m27.207548608s) [upload=197386]: Done uploading segments... Dec 13 13:20:26.379861196 (189h52m27.207569587s) [upload=197386]: Upload finished. err: <nil> Duration: 3.610845401s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 3.465888277s trace: https://jaeger.dev.storj.io/trace/12dbee1725a89af6 FAIL (2025-12-13 13:29:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=197386]: Uploading piece... Dec 13 13:20:22.157523997 (189h52m22.985232392s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157563526 (189h52m22.985271946s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157782226 (189h52m22.985490616s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158220136 (189h52m22.98592853s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158561593 (189h52m22.986269971s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158833133 (189h52m22.986541524s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159110305 (189h52m22.986818699s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159379461 (189h52m22.987087838s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159627778 (189h52m22.987336156s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.588165740 (189h52m23.41587413s) [upload=197386]: Next returned nil segment Dec 13 13:20:22.588188840 (189h52m23.415897231s) [upload=197386]: Waiting for error group managing segments... Dec 13 13:20:22.787355936 (189h52m23.615064332s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.793561088 (189h52m23.621269481s) [noise=true , address=185.167.204.205:4013 , piece_id=QNJLAFTMBQPQF7IBMT6EKMBEL5KX7VHU3L4L3ALPXYNUV2MZ43GA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.801170609 (189h52m23.628879002s) [noise=true , address=185.167.204.206:4068 , piece_id=P2FTDV3XW34Z6M73CWIURQCCQ6H5NXDNJWKBVN3GRQTDV65QLP5A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.812199494 (189h52m23.63990789s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813259194 (189h52m23.640967588s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813831001 (189h52m23.641539395s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814310957 (189h52m23.642019347s) [noise=true , address=185.167.204.205:4001 , piece_id=WXEZMLB4ZM2FFOB4JMCIF2X6FTB72I3EWWRTDLIZ6SXCMUTS5HGQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814961478 (189h52m23.64266987s) [noise=true , address=185.167.204.206:4046 , piece_id=KLHWCCQLK3CAOIMW57WAFZHNMECYLGA4VRZE3LXRUJRQCKAGGBKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.815469120 (189h52m23.643177512s) [noise=true , address=185.167.204.206:4054 , piece_id=ZPZSVEBAXMRI7732XKIHUGMU2WGGCNKE34U2FZ3NHJLKCMYBIUXQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.817780190 (189h52m23.645488584s) [noise=true , address=185.167.204.206:4045 , piece_id=VQ37X765RRBAVPI4HIAPJET7VRCI2FE7FRHB45II6ETFYDYMKSTQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818186706 (189h52m23.645895101s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818345893 (189h52m23.646054284s) [noise=true , address=185.167.204.205:4014 , piece_id=4U6VXOGIXKAV7NJBQBD2V6ZVWAUIT557OP7AUG332UWVS5LZFSPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.844792090 (189h52m23.672516847s) [noise=true , address=185.167.204.206:4051 , piece_id=A4VZAGPIVLUVOVOIGGIK7CEDRPUSV34XMUL5TFL5KFJNICAWSJ4Q, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.354540799 (189h52m25.182249185s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.369904036 (189h52m25.197612417s) [noise=true , address=81.171.14.70:4039 , piece_id=YV7FP35KY6TXYDKBEOV6RCHCSJWR4TQCFEB43GK3AZP4C776BCZQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.378758619 (189h52m25.206467012s) [noise=true , address=81.171.14.70:4047 , piece_id=EQFLTLGTZBKQ47MWFLSBCEHDKAWMZSGJHZPBAN67EANSP2LR7HYQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.384761037 (189h52m25.212469431s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.495107931 (189h52m25.322816323s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.833114410 (189h52m25.660822801s) [noise=true , address=89.149.201.147:4009 , piece_id=SAES2PXHWVMC22L3DKLBYHAI3ZHKYKGYOULL4ZZ2AB2U53B5URVA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.076006180 (189h52m25.903714572s) [noise=true , address=89.149.201.147:4008 , piece_id=HL2EBKXU5RWRNZP7D3MUOTRJK2YJTADFQS6C4IF7CGEUF3PLOIMQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.109256616 (189h52m25.936964999s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.121296573 (189h52m25.949004965s) [noise=true , address=81.171.14.70:4034 , piece_id=XZ72ODCL37AYLPMMFTCT4NJROVRCLTEM2XN6SQVG3G5NTZRJDWDA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.144861937 (189h52m25.972570318s) [noise=true , address=89.149.201.147:4012 , piece_id=5LDACT5ZGR54I4LNLZQHNXUY2VZQ5XIWI5AYUM5CZHCOOQIBGPRQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.859441967 (189h52m26.687150358s) [noise=true , address=81.171.14.70:4026 , piece_id=R7B2VE64HYLI7A2XVWDFTIIWAVLT6I6E75ZCAMTDH77R7BW2Q7IQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.932179545 (189h52m26.759887944s) [noise=true , address=89.149.201.147:4001 , piece_id=GDNVHYCJJDXLUDF77L6BP4IMIMR6KRZ2FTZJ6BG5TMAZGIW75PHQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228904886 (189h52m27.056613283s) [noise=true , address=89.149.201.147:4024 , piece_id=7MCMBHNW4L7JDB4ATRHAM3GWVFAYH34LWFNX5O7C6EXIB34BNWRA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228954088 (189h52m27.056662491s) [seg_pos={0 0}, upload=197386]: Segment reached optimal threshold of 26 pieces. Dec 13 13:20:26.229843002 (189h52m27.057551404s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: read tcp 159.223.143.39:53228->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.229857857 (189h52m27.05756626s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230208353 (189h52m27.057916754s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7): protocol: read tcp 159.223.143.39:45318->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.230219436 (189h52m27.057927837s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230249796 (189h52m27.057958176s) [seg_pos={0 0}, upload=197386]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 13 13:20:26.230536147 (189h52m27.05824454s) [upload=197386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 13 13:20:26.230543508 (189h52m27.058251899s) [upload=197386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 13 13:20:26.379840216 (189h52m27.207548608s) [upload=197386]: Done uploading segments... Dec 13 13:20:26.379861196 (189h52m27.207569587s) [upload=197386]: Upload finished. err: <nil> Duration: 3.465914225s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 3.465351118s trace: https://jaeger.dev.storj.io/trace/12dbee1725a89af6 FAIL (2025-12-13 13:28:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=197386]: Uploading piece... Dec 13 13:20:22.157523997 (189h52m22.985232392s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157563526 (189h52m22.985271946s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157782226 (189h52m22.985490616s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158220136 (189h52m22.98592853s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158561593 (189h52m22.986269971s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158833133 (189h52m22.986541524s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159110305 (189h52m22.986818699s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159379461 (189h52m22.987087838s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159627778 (189h52m22.987336156s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.588165740 (189h52m23.41587413s) [upload=197386]: Next returned nil segment Dec 13 13:20:22.588188840 (189h52m23.415897231s) [upload=197386]: Waiting for error group managing segments... Dec 13 13:20:22.787355936 (189h52m23.615064332s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.793561088 (189h52m23.621269481s) [noise=true , address=185.167.204.205:4013 , piece_id=QNJLAFTMBQPQF7IBMT6EKMBEL5KX7VHU3L4L3ALPXYNUV2MZ43GA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.801170609 (189h52m23.628879002s) [noise=true , address=185.167.204.206:4068 , piece_id=P2FTDV3XW34Z6M73CWIURQCCQ6H5NXDNJWKBVN3GRQTDV65QLP5A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.812199494 (189h52m23.63990789s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813259194 (189h52m23.640967588s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813831001 (189h52m23.641539395s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814310957 (189h52m23.642019347s) [noise=true , address=185.167.204.205:4001 , piece_id=WXEZMLB4ZM2FFOB4JMCIF2X6FTB72I3EWWRTDLIZ6SXCMUTS5HGQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814961478 (189h52m23.64266987s) [noise=true , address=185.167.204.206:4046 , piece_id=KLHWCCQLK3CAOIMW57WAFZHNMECYLGA4VRZE3LXRUJRQCKAGGBKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.815469120 (189h52m23.643177512s) [noise=true , address=185.167.204.206:4054 , piece_id=ZPZSVEBAXMRI7732XKIHUGMU2WGGCNKE34U2FZ3NHJLKCMYBIUXQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.817780190 (189h52m23.645488584s) [noise=true , address=185.167.204.206:4045 , piece_id=VQ37X765RRBAVPI4HIAPJET7VRCI2FE7FRHB45II6ETFYDYMKSTQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818186706 (189h52m23.645895101s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818345893 (189h52m23.646054284s) [noise=true , address=185.167.204.205:4014 , piece_id=4U6VXOGIXKAV7NJBQBD2V6ZVWAUIT557OP7AUG332UWVS5LZFSPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.844792090 (189h52m23.672516847s) [noise=true , address=185.167.204.206:4051 , piece_id=A4VZAGPIVLUVOVOIGGIK7CEDRPUSV34XMUL5TFL5KFJNICAWSJ4Q, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.354540799 (189h52m25.182249185s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.369904036 (189h52m25.197612417s) [noise=true , address=81.171.14.70:4039 , piece_id=YV7FP35KY6TXYDKBEOV6RCHCSJWR4TQCFEB43GK3AZP4C776BCZQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.378758619 (189h52m25.206467012s) [noise=true , address=81.171.14.70:4047 , piece_id=EQFLTLGTZBKQ47MWFLSBCEHDKAWMZSGJHZPBAN67EANSP2LR7HYQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.384761037 (189h52m25.212469431s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.495107931 (189h52m25.322816323s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.833114410 (189h52m25.660822801s) [noise=true , address=89.149.201.147:4009 , piece_id=SAES2PXHWVMC22L3DKLBYHAI3ZHKYKGYOULL4ZZ2AB2U53B5URVA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.076006180 (189h52m25.903714572s) [noise=true , address=89.149.201.147:4008 , piece_id=HL2EBKXU5RWRNZP7D3MUOTRJK2YJTADFQS6C4IF7CGEUF3PLOIMQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.109256616 (189h52m25.936964999s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.121296573 (189h52m25.949004965s) [noise=true , address=81.171.14.70:4034 , piece_id=XZ72ODCL37AYLPMMFTCT4NJROVRCLTEM2XN6SQVG3G5NTZRJDWDA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.144861937 (189h52m25.972570318s) [noise=true , address=89.149.201.147:4012 , piece_id=5LDACT5ZGR54I4LNLZQHNXUY2VZQ5XIWI5AYUM5CZHCOOQIBGPRQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.859441967 (189h52m26.687150358s) [noise=true , address=81.171.14.70:4026 , piece_id=R7B2VE64HYLI7A2XVWDFTIIWAVLT6I6E75ZCAMTDH77R7BW2Q7IQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.932179545 (189h52m26.759887944s) [noise=true , address=89.149.201.147:4001 , piece_id=GDNVHYCJJDXLUDF77L6BP4IMIMR6KRZ2FTZJ6BG5TMAZGIW75PHQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228904886 (189h52m27.056613283s) [noise=true , address=89.149.201.147:4024 , piece_id=7MCMBHNW4L7JDB4ATRHAM3GWVFAYH34LWFNX5O7C6EXIB34BNWRA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228954088 (189h52m27.056662491s) [seg_pos={0 0}, upload=197386]: Segment reached optimal threshold of 26 pieces. Dec 13 13:20:26.229843002 (189h52m27.057551404s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: read tcp 159.223.143.39:53228->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.229857857 (189h52m27.05756626s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230208353 (189h52m27.057916754s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7): protocol: read tcp 159.223.143.39:45318->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.230219436 (189h52m27.057927837s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230249796 (189h52m27.057958176s) [seg_pos={0 0}, upload=197386]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 13 13:20:26.230536147 (189h52m27.05824454s) [upload=197386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 13 13:20:26.230543508 (189h52m27.058251899s) [upload=197386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 13 13:20:26.379840216 (189h52m27.207548608s) [upload=197386]: Done uploading segments... Dec 13 13:20:26.379861196 (189h52m27.207569587s) [upload=197386]: Upload finished. err: <nil> Duration: 3.465379217s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 3.725626549s trace: https://jaeger.dev.storj.io/trace/12dbee1725a89af6 FAIL (2025-12-13 13:28:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=197386]: Uploading piece... Dec 13 13:20:22.157523997 (189h52m22.985232392s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157563526 (189h52m22.985271946s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157782226 (189h52m22.985490616s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158220136 (189h52m22.98592853s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158561593 (189h52m22.986269971s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158833133 (189h52m22.986541524s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159110305 (189h52m22.986818699s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159379461 (189h52m22.987087838s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159627778 (189h52m22.987336156s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.588165740 (189h52m23.41587413s) [upload=197386]: Next returned nil segment Dec 13 13:20:22.588188840 (189h52m23.415897231s) [upload=197386]: Waiting for error group managing segments... Dec 13 13:20:22.787355936 (189h52m23.615064332s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.793561088 (189h52m23.621269481s) [noise=true , address=185.167.204.205:4013 , piece_id=QNJLAFTMBQPQF7IBMT6EKMBEL5KX7VHU3L4L3ALPXYNUV2MZ43GA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.801170609 (189h52m23.628879002s) [noise=true , address=185.167.204.206:4068 , piece_id=P2FTDV3XW34Z6M73CWIURQCCQ6H5NXDNJWKBVN3GRQTDV65QLP5A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.812199494 (189h52m23.63990789s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813259194 (189h52m23.640967588s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813831001 (189h52m23.641539395s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814310957 (189h52m23.642019347s) [noise=true , address=185.167.204.205:4001 , piece_id=WXEZMLB4ZM2FFOB4JMCIF2X6FTB72I3EWWRTDLIZ6SXCMUTS5HGQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814961478 (189h52m23.64266987s) [noise=true , address=185.167.204.206:4046 , piece_id=KLHWCCQLK3CAOIMW57WAFZHNMECYLGA4VRZE3LXRUJRQCKAGGBKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.815469120 (189h52m23.643177512s) [noise=true , address=185.167.204.206:4054 , piece_id=ZPZSVEBAXMRI7732XKIHUGMU2WGGCNKE34U2FZ3NHJLKCMYBIUXQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.817780190 (189h52m23.645488584s) [noise=true , address=185.167.204.206:4045 , piece_id=VQ37X765RRBAVPI4HIAPJET7VRCI2FE7FRHB45II6ETFYDYMKSTQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818186706 (189h52m23.645895101s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818345893 (189h52m23.646054284s) [noise=true , address=185.167.204.205:4014 , piece_id=4U6VXOGIXKAV7NJBQBD2V6ZVWAUIT557OP7AUG332UWVS5LZFSPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.844792090 (189h52m23.672516847s) [noise=true , address=185.167.204.206:4051 , piece_id=A4VZAGPIVLUVOVOIGGIK7CEDRPUSV34XMUL5TFL5KFJNICAWSJ4Q, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.354540799 (189h52m25.182249185s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.369904036 (189h52m25.197612417s) [noise=true , address=81.171.14.70:4039 , piece_id=YV7FP35KY6TXYDKBEOV6RCHCSJWR4TQCFEB43GK3AZP4C776BCZQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.378758619 (189h52m25.206467012s) [noise=true , address=81.171.14.70:4047 , piece_id=EQFLTLGTZBKQ47MWFLSBCEHDKAWMZSGJHZPBAN67EANSP2LR7HYQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.384761037 (189h52m25.212469431s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.495107931 (189h52m25.322816323s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.833114410 (189h52m25.660822801s) [noise=true , address=89.149.201.147:4009 , piece_id=SAES2PXHWVMC22L3DKLBYHAI3ZHKYKGYOULL4ZZ2AB2U53B5URVA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.076006180 (189h52m25.903714572s) [noise=true , address=89.149.201.147:4008 , piece_id=HL2EBKXU5RWRNZP7D3MUOTRJK2YJTADFQS6C4IF7CGEUF3PLOIMQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.109256616 (189h52m25.936964999s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.121296573 (189h52m25.949004965s) [noise=true , address=81.171.14.70:4034 , piece_id=XZ72ODCL37AYLPMMFTCT4NJROVRCLTEM2XN6SQVG3G5NTZRJDWDA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.144861937 (189h52m25.972570318s) [noise=true , address=89.149.201.147:4012 , piece_id=5LDACT5ZGR54I4LNLZQHNXUY2VZQ5XIWI5AYUM5CZHCOOQIBGPRQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.859441967 (189h52m26.687150358s) [noise=true , address=81.171.14.70:4026 , piece_id=R7B2VE64HYLI7A2XVWDFTIIWAVLT6I6E75ZCAMTDH77R7BW2Q7IQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.932179545 (189h52m26.759887944s) [noise=true , address=89.149.201.147:4001 , piece_id=GDNVHYCJJDXLUDF77L6BP4IMIMR6KRZ2FTZJ6BG5TMAZGIW75PHQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228904886 (189h52m27.056613283s) [noise=true , address=89.149.201.147:4024 , piece_id=7MCMBHNW4L7JDB4ATRHAM3GWVFAYH34LWFNX5O7C6EXIB34BNWRA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228954088 (189h52m27.056662491s) [seg_pos={0 0}, upload=197386]: Segment reached optimal threshold of 26 pieces. Dec 13 13:20:26.229843002 (189h52m27.057551404s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: read tcp 159.223.143.39:53228->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.229857857 (189h52m27.05756626s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230208353 (189h52m27.057916754s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7): protocol: read tcp 159.223.143.39:45318->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.230219436 (189h52m27.057927837s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230249796 (189h52m27.057958176s) [seg_pos={0 0}, upload=197386]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 13 13:20:26.230536147 (189h52m27.05824454s) [upload=197386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 13 13:20:26.230543508 (189h52m27.058251899s) [upload=197386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 13 13:20:26.379840216 (189h52m27.207548608s) [upload=197386]: Done uploading segments... Dec 13 13:20:26.379861196 (189h52m27.207569587s) [upload=197386]: Upload finished. err: <nil> Duration: 3.72566017s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 4.080176815s trace: https://jaeger.dev.storj.io/trace/12dbee1725a89af6 FAIL (2025-12-13 13:27:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=197386]: Uploading piece... Dec 13 13:20:22.157523997 (189h52m22.985232392s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157563526 (189h52m22.985271946s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157782226 (189h52m22.985490616s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158220136 (189h52m22.98592853s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158561593 (189h52m22.986269971s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158833133 (189h52m22.986541524s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159110305 (189h52m22.986818699s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159379461 (189h52m22.987087838s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159627778 (189h52m22.987336156s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.588165740 (189h52m23.41587413s) [upload=197386]: Next returned nil segment Dec 13 13:20:22.588188840 (189h52m23.415897231s) [upload=197386]: Waiting for error group managing segments... Dec 13 13:20:22.787355936 (189h52m23.615064332s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.793561088 (189h52m23.621269481s) [noise=true , address=185.167.204.205:4013 , piece_id=QNJLAFTMBQPQF7IBMT6EKMBEL5KX7VHU3L4L3ALPXYNUV2MZ43GA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.801170609 (189h52m23.628879002s) [noise=true , address=185.167.204.206:4068 , piece_id=P2FTDV3XW34Z6M73CWIURQCCQ6H5NXDNJWKBVN3GRQTDV65QLP5A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.812199494 (189h52m23.63990789s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813259194 (189h52m23.640967588s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813831001 (189h52m23.641539395s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814310957 (189h52m23.642019347s) [noise=true , address=185.167.204.205:4001 , piece_id=WXEZMLB4ZM2FFOB4JMCIF2X6FTB72I3EWWRTDLIZ6SXCMUTS5HGQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814961478 (189h52m23.64266987s) [noise=true , address=185.167.204.206:4046 , piece_id=KLHWCCQLK3CAOIMW57WAFZHNMECYLGA4VRZE3LXRUJRQCKAGGBKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.815469120 (189h52m23.643177512s) [noise=true , address=185.167.204.206:4054 , piece_id=ZPZSVEBAXMRI7732XKIHUGMU2WGGCNKE34U2FZ3NHJLKCMYBIUXQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.817780190 (189h52m23.645488584s) [noise=true , address=185.167.204.206:4045 , piece_id=VQ37X765RRBAVPI4HIAPJET7VRCI2FE7FRHB45II6ETFYDYMKSTQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818186706 (189h52m23.645895101s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818345893 (189h52m23.646054284s) [noise=true , address=185.167.204.205:4014 , piece_id=4U6VXOGIXKAV7NJBQBD2V6ZVWAUIT557OP7AUG332UWVS5LZFSPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.844792090 (189h52m23.672516847s) [noise=true , address=185.167.204.206:4051 , piece_id=A4VZAGPIVLUVOVOIGGIK7CEDRPUSV34XMUL5TFL5KFJNICAWSJ4Q, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.354540799 (189h52m25.182249185s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.369904036 (189h52m25.197612417s) [noise=true , address=81.171.14.70:4039 , piece_id=YV7FP35KY6TXYDKBEOV6RCHCSJWR4TQCFEB43GK3AZP4C776BCZQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.378758619 (189h52m25.206467012s) [noise=true , address=81.171.14.70:4047 , piece_id=EQFLTLGTZBKQ47MWFLSBCEHDKAWMZSGJHZPBAN67EANSP2LR7HYQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.384761037 (189h52m25.212469431s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.495107931 (189h52m25.322816323s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.833114410 (189h52m25.660822801s) [noise=true , address=89.149.201.147:4009 , piece_id=SAES2PXHWVMC22L3DKLBYHAI3ZHKYKGYOULL4ZZ2AB2U53B5URVA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.076006180 (189h52m25.903714572s) [noise=true , address=89.149.201.147:4008 , piece_id=HL2EBKXU5RWRNZP7D3MUOTRJK2YJTADFQS6C4IF7CGEUF3PLOIMQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.109256616 (189h52m25.936964999s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.121296573 (189h52m25.949004965s) [noise=true , address=81.171.14.70:4034 , piece_id=XZ72ODCL37AYLPMMFTCT4NJROVRCLTEM2XN6SQVG3G5NTZRJDWDA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.144861937 (189h52m25.972570318s) [noise=true , address=89.149.201.147:4012 , piece_id=5LDACT5ZGR54I4LNLZQHNXUY2VZQ5XIWI5AYUM5CZHCOOQIBGPRQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.859441967 (189h52m26.687150358s) [noise=true , address=81.171.14.70:4026 , piece_id=R7B2VE64HYLI7A2XVWDFTIIWAVLT6I6E75ZCAMTDH77R7BW2Q7IQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.932179545 (189h52m26.759887944s) [noise=true , address=89.149.201.147:4001 , piece_id=GDNVHYCJJDXLUDF77L6BP4IMIMR6KRZ2FTZJ6BG5TMAZGIW75PHQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228904886 (189h52m27.056613283s) [noise=true , address=89.149.201.147:4024 , piece_id=7MCMBHNW4L7JDB4ATRHAM3GWVFAYH34LWFNX5O7C6EXIB34BNWRA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228954088 (189h52m27.056662491s) [seg_pos={0 0}, upload=197386]: Segment reached optimal threshold of 26 pieces. Dec 13 13:20:26.229843002 (189h52m27.057551404s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: read tcp 159.223.143.39:53228->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.229857857 (189h52m27.05756626s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230208353 (189h52m27.057916754s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7): protocol: read tcp 159.223.143.39:45318->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.230219436 (189h52m27.057927837s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230249796 (189h52m27.057958176s) [seg_pos={0 0}, upload=197386]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 13 13:20:26.230536147 (189h52m27.05824454s) [upload=197386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 13 13:20:26.230543508 (189h52m27.058251899s) [upload=197386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 13 13:20:26.379840216 (189h52m27.207548608s) [upload=197386]: Done uploading segments... Dec 13 13:20:26.379861196 (189h52m27.207569587s) [upload=197386]: Upload finished. err: <nil> Duration: 4.080200949s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 4.370125087s trace: https://jaeger.dev.storj.io/trace/12dbee1725a89af6 FAIL (2025-12-13 13:26:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=197386]: Uploading piece... Dec 13 13:20:22.157523997 (189h52m22.985232392s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157563526 (189h52m22.985271946s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157782226 (189h52m22.985490616s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158220136 (189h52m22.98592853s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158561593 (189h52m22.986269971s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158833133 (189h52m22.986541524s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159110305 (189h52m22.986818699s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159379461 (189h52m22.987087838s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159627778 (189h52m22.987336156s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.588165740 (189h52m23.41587413s) [upload=197386]: Next returned nil segment Dec 13 13:20:22.588188840 (189h52m23.415897231s) [upload=197386]: Waiting for error group managing segments... Dec 13 13:20:22.787355936 (189h52m23.615064332s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.793561088 (189h52m23.621269481s) [noise=true , address=185.167.204.205:4013 , piece_id=QNJLAFTMBQPQF7IBMT6EKMBEL5KX7VHU3L4L3ALPXYNUV2MZ43GA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.801170609 (189h52m23.628879002s) [noise=true , address=185.167.204.206:4068 , piece_id=P2FTDV3XW34Z6M73CWIURQCCQ6H5NXDNJWKBVN3GRQTDV65QLP5A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.812199494 (189h52m23.63990789s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813259194 (189h52m23.640967588s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813831001 (189h52m23.641539395s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814310957 (189h52m23.642019347s) [noise=true , address=185.167.204.205:4001 , piece_id=WXEZMLB4ZM2FFOB4JMCIF2X6FTB72I3EWWRTDLIZ6SXCMUTS5HGQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814961478 (189h52m23.64266987s) [noise=true , address=185.167.204.206:4046 , piece_id=KLHWCCQLK3CAOIMW57WAFZHNMECYLGA4VRZE3LXRUJRQCKAGGBKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.815469120 (189h52m23.643177512s) [noise=true , address=185.167.204.206:4054 , piece_id=ZPZSVEBAXMRI7732XKIHUGMU2WGGCNKE34U2FZ3NHJLKCMYBIUXQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.817780190 (189h52m23.645488584s) [noise=true , address=185.167.204.206:4045 , piece_id=VQ37X765RRBAVPI4HIAPJET7VRCI2FE7FRHB45II6ETFYDYMKSTQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818186706 (189h52m23.645895101s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818345893 (189h52m23.646054284s) [noise=true , address=185.167.204.205:4014 , piece_id=4U6VXOGIXKAV7NJBQBD2V6ZVWAUIT557OP7AUG332UWVS5LZFSPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.844792090 (189h52m23.672516847s) [noise=true , address=185.167.204.206:4051 , piece_id=A4VZAGPIVLUVOVOIGGIK7CEDRPUSV34XMUL5TFL5KFJNICAWSJ4Q, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.354540799 (189h52m25.182249185s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.369904036 (189h52m25.197612417s) [noise=true , address=81.171.14.70:4039 , piece_id=YV7FP35KY6TXYDKBEOV6RCHCSJWR4TQCFEB43GK3AZP4C776BCZQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.378758619 (189h52m25.206467012s) [noise=true , address=81.171.14.70:4047 , piece_id=EQFLTLGTZBKQ47MWFLSBCEHDKAWMZSGJHZPBAN67EANSP2LR7HYQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.384761037 (189h52m25.212469431s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.495107931 (189h52m25.322816323s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.833114410 (189h52m25.660822801s) [noise=true , address=89.149.201.147:4009 , piece_id=SAES2PXHWVMC22L3DKLBYHAI3ZHKYKGYOULL4ZZ2AB2U53B5URVA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.076006180 (189h52m25.903714572s) [noise=true , address=89.149.201.147:4008 , piece_id=HL2EBKXU5RWRNZP7D3MUOTRJK2YJTADFQS6C4IF7CGEUF3PLOIMQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.109256616 (189h52m25.936964999s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.121296573 (189h52m25.949004965s) [noise=true , address=81.171.14.70:4034 , piece_id=XZ72ODCL37AYLPMMFTCT4NJROVRCLTEM2XN6SQVG3G5NTZRJDWDA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.144861937 (189h52m25.972570318s) [noise=true , address=89.149.201.147:4012 , piece_id=5LDACT5ZGR54I4LNLZQHNXUY2VZQ5XIWI5AYUM5CZHCOOQIBGPRQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.859441967 (189h52m26.687150358s) [noise=true , address=81.171.14.70:4026 , piece_id=R7B2VE64HYLI7A2XVWDFTIIWAVLT6I6E75ZCAMTDH77R7BW2Q7IQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.932179545 (189h52m26.759887944s) [noise=true , address=89.149.201.147:4001 , piece_id=GDNVHYCJJDXLUDF77L6BP4IMIMR6KRZ2FTZJ6BG5TMAZGIW75PHQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228904886 (189h52m27.056613283s) [noise=true , address=89.149.201.147:4024 , piece_id=7MCMBHNW4L7JDB4ATRHAM3GWVFAYH34LWFNX5O7C6EXIB34BNWRA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228954088 (189h52m27.056662491s) [seg_pos={0 0}, upload=197386]: Segment reached optimal threshold of 26 pieces. Dec 13 13:20:26.229843002 (189h52m27.057551404s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: read tcp 159.223.143.39:53228->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.229857857 (189h52m27.05756626s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230208353 (189h52m27.057916754s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7): protocol: read tcp 159.223.143.39:45318->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.230219436 (189h52m27.057927837s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230249796 (189h52m27.057958176s) [seg_pos={0 0}, upload=197386]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 13 13:20:26.230536147 (189h52m27.05824454s) [upload=197386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 13 13:20:26.230543508 (189h52m27.058251899s) [upload=197386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 13 13:20:26.379840216 (189h52m27.207548608s) [upload=197386]: Done uploading segments... Dec 13 13:20:26.379861196 (189h52m27.207569587s) [upload=197386]: Upload finished. err: <nil> Duration: 4.370155591s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 4.213951853s trace: https://jaeger.dev.storj.io/trace/12dbee1725a89af6 FAIL (2025-12-13 13:25:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=197386]: Uploading piece... Dec 13 13:20:22.157523997 (189h52m22.985232392s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157563526 (189h52m22.985271946s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157782226 (189h52m22.985490616s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158220136 (189h52m22.98592853s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158561593 (189h52m22.986269971s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158833133 (189h52m22.986541524s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159110305 (189h52m22.986818699s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159379461 (189h52m22.987087838s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159627778 (189h52m22.987336156s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.588165740 (189h52m23.41587413s) [upload=197386]: Next returned nil segment Dec 13 13:20:22.588188840 (189h52m23.415897231s) [upload=197386]: Waiting for error group managing segments... Dec 13 13:20:22.787355936 (189h52m23.615064332s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.793561088 (189h52m23.621269481s) [noise=true , address=185.167.204.205:4013 , piece_id=QNJLAFTMBQPQF7IBMT6EKMBEL5KX7VHU3L4L3ALPXYNUV2MZ43GA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.801170609 (189h52m23.628879002s) [noise=true , address=185.167.204.206:4068 , piece_id=P2FTDV3XW34Z6M73CWIURQCCQ6H5NXDNJWKBVN3GRQTDV65QLP5A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.812199494 (189h52m23.63990789s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813259194 (189h52m23.640967588s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813831001 (189h52m23.641539395s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814310957 (189h52m23.642019347s) [noise=true , address=185.167.204.205:4001 , piece_id=WXEZMLB4ZM2FFOB4JMCIF2X6FTB72I3EWWRTDLIZ6SXCMUTS5HGQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814961478 (189h52m23.64266987s) [noise=true , address=185.167.204.206:4046 , piece_id=KLHWCCQLK3CAOIMW57WAFZHNMECYLGA4VRZE3LXRUJRQCKAGGBKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.815469120 (189h52m23.643177512s) [noise=true , address=185.167.204.206:4054 , piece_id=ZPZSVEBAXMRI7732XKIHUGMU2WGGCNKE34U2FZ3NHJLKCMYBIUXQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.817780190 (189h52m23.645488584s) [noise=true , address=185.167.204.206:4045 , piece_id=VQ37X765RRBAVPI4HIAPJET7VRCI2FE7FRHB45II6ETFYDYMKSTQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818186706 (189h52m23.645895101s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818345893 (189h52m23.646054284s) [noise=true , address=185.167.204.205:4014 , piece_id=4U6VXOGIXKAV7NJBQBD2V6ZVWAUIT557OP7AUG332UWVS5LZFSPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.844792090 (189h52m23.672516847s) [noise=true , address=185.167.204.206:4051 , piece_id=A4VZAGPIVLUVOVOIGGIK7CEDRPUSV34XMUL5TFL5KFJNICAWSJ4Q, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.354540799 (189h52m25.182249185s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.369904036 (189h52m25.197612417s) [noise=true , address=81.171.14.70:4039 , piece_id=YV7FP35KY6TXYDKBEOV6RCHCSJWR4TQCFEB43GK3AZP4C776BCZQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.378758619 (189h52m25.206467012s) [noise=true , address=81.171.14.70:4047 , piece_id=EQFLTLGTZBKQ47MWFLSBCEHDKAWMZSGJHZPBAN67EANSP2LR7HYQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.384761037 (189h52m25.212469431s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.495107931 (189h52m25.322816323s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.833114410 (189h52m25.660822801s) [noise=true , address=89.149.201.147:4009 , piece_id=SAES2PXHWVMC22L3DKLBYHAI3ZHKYKGYOULL4ZZ2AB2U53B5URVA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.076006180 (189h52m25.903714572s) [noise=true , address=89.149.201.147:4008 , piece_id=HL2EBKXU5RWRNZP7D3MUOTRJK2YJTADFQS6C4IF7CGEUF3PLOIMQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.109256616 (189h52m25.936964999s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.121296573 (189h52m25.949004965s) [noise=true , address=81.171.14.70:4034 , piece_id=XZ72ODCL37AYLPMMFTCT4NJROVRCLTEM2XN6SQVG3G5NTZRJDWDA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.144861937 (189h52m25.972570318s) [noise=true , address=89.149.201.147:4012 , piece_id=5LDACT5ZGR54I4LNLZQHNXUY2VZQ5XIWI5AYUM5CZHCOOQIBGPRQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.859441967 (189h52m26.687150358s) [noise=true , address=81.171.14.70:4026 , piece_id=R7B2VE64HYLI7A2XVWDFTIIWAVLT6I6E75ZCAMTDH77R7BW2Q7IQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.932179545 (189h52m26.759887944s) [noise=true , address=89.149.201.147:4001 , piece_id=GDNVHYCJJDXLUDF77L6BP4IMIMR6KRZ2FTZJ6BG5TMAZGIW75PHQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228904886 (189h52m27.056613283s) [noise=true , address=89.149.201.147:4024 , piece_id=7MCMBHNW4L7JDB4ATRHAM3GWVFAYH34LWFNX5O7C6EXIB34BNWRA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228954088 (189h52m27.056662491s) [seg_pos={0 0}, upload=197386]: Segment reached optimal threshold of 26 pieces. Dec 13 13:20:26.229843002 (189h52m27.057551404s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: read tcp 159.223.143.39:53228->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.229857857 (189h52m27.05756626s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230208353 (189h52m27.057916754s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7): protocol: read tcp 159.223.143.39:45318->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.230219436 (189h52m27.057927837s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230249796 (189h52m27.057958176s) [seg_pos={0 0}, upload=197386]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 13 13:20:26.230536147 (189h52m27.05824454s) [upload=197386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 13 13:20:26.230543508 (189h52m27.058251899s) [upload=197386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 13 13:20:26.379840216 (189h52m27.207548608s) [upload=197386]: Done uploading segments... Dec 13 13:20:26.379861196 (189h52m27.207569587s) [upload=197386]: Upload finished. err: <nil> Duration: 4.213977858s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 35.627021781s trace: https://jaeger.dev.storj.io/trace/12dbee1725a89af6 FAIL (2025-12-13 13:24:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=197386]: Uploading piece... Dec 13 13:20:22.157523997 (189h52m22.985232392s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157563526 (189h52m22.985271946s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157782226 (189h52m22.985490616s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158220136 (189h52m22.98592853s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158561593 (189h52m22.986269971s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158833133 (189h52m22.986541524s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159110305 (189h52m22.986818699s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159379461 (189h52m22.987087838s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159627778 (189h52m22.987336156s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.588165740 (189h52m23.41587413s) [upload=197386]: Next returned nil segment Dec 13 13:20:22.588188840 (189h52m23.415897231s) [upload=197386]: Waiting for error group managing segments... Dec 13 13:20:22.787355936 (189h52m23.615064332s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.793561088 (189h52m23.621269481s) [noise=true , address=185.167.204.205:4013 , piece_id=QNJLAFTMBQPQF7IBMT6EKMBEL5KX7VHU3L4L3ALPXYNUV2MZ43GA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.801170609 (189h52m23.628879002s) [noise=true , address=185.167.204.206:4068 , piece_id=P2FTDV3XW34Z6M73CWIURQCCQ6H5NXDNJWKBVN3GRQTDV65QLP5A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.812199494 (189h52m23.63990789s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813259194 (189h52m23.640967588s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813831001 (189h52m23.641539395s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814310957 (189h52m23.642019347s) [noise=true , address=185.167.204.205:4001 , piece_id=WXEZMLB4ZM2FFOB4JMCIF2X6FTB72I3EWWRTDLIZ6SXCMUTS5HGQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814961478 (189h52m23.64266987s) [noise=true , address=185.167.204.206:4046 , piece_id=KLHWCCQLK3CAOIMW57WAFZHNMECYLGA4VRZE3LXRUJRQCKAGGBKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.815469120 (189h52m23.643177512s) [noise=true , address=185.167.204.206:4054 , piece_id=ZPZSVEBAXMRI7732XKIHUGMU2WGGCNKE34U2FZ3NHJLKCMYBIUXQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.817780190 (189h52m23.645488584s) [noise=true , address=185.167.204.206:4045 , piece_id=VQ37X765RRBAVPI4HIAPJET7VRCI2FE7FRHB45II6ETFYDYMKSTQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818186706 (189h52m23.645895101s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818345893 (189h52m23.646054284s) [noise=true , address=185.167.204.205:4014 , piece_id=4U6VXOGIXKAV7NJBQBD2V6ZVWAUIT557OP7AUG332UWVS5LZFSPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.844792090 (189h52m23.672516847s) [noise=true , address=185.167.204.206:4051 , piece_id=A4VZAGPIVLUVOVOIGGIK7CEDRPUSV34XMUL5TFL5KFJNICAWSJ4Q, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.354540799 (189h52m25.182249185s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.369904036 (189h52m25.197612417s) [noise=true , address=81.171.14.70:4039 , piece_id=YV7FP35KY6TXYDKBEOV6RCHCSJWR4TQCFEB43GK3AZP4C776BCZQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.378758619 (189h52m25.206467012s) [noise=true , address=81.171.14.70:4047 , piece_id=EQFLTLGTZBKQ47MWFLSBCEHDKAWMZSGJHZPBAN67EANSP2LR7HYQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.384761037 (189h52m25.212469431s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.495107931 (189h52m25.322816323s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.833114410 (189h52m25.660822801s) [noise=true , address=89.149.201.147:4009 , piece_id=SAES2PXHWVMC22L3DKLBYHAI3ZHKYKGYOULL4ZZ2AB2U53B5URVA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.076006180 (189h52m25.903714572s) [noise=true , address=89.149.201.147:4008 , piece_id=HL2EBKXU5RWRNZP7D3MUOTRJK2YJTADFQS6C4IF7CGEUF3PLOIMQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.109256616 (189h52m25.936964999s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.121296573 (189h52m25.949004965s) [noise=true , address=81.171.14.70:4034 , piece_id=XZ72ODCL37AYLPMMFTCT4NJROVRCLTEM2XN6SQVG3G5NTZRJDWDA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.144861937 (189h52m25.972570318s) [noise=true , address=89.149.201.147:4012 , piece_id=5LDACT5ZGR54I4LNLZQHNXUY2VZQ5XIWI5AYUM5CZHCOOQIBGPRQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.859441967 (189h52m26.687150358s) [noise=true , address=81.171.14.70:4026 , piece_id=R7B2VE64HYLI7A2XVWDFTIIWAVLT6I6E75ZCAMTDH77R7BW2Q7IQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.932179545 (189h52m26.759887944s) [noise=true , address=89.149.201.147:4001 , piece_id=GDNVHYCJJDXLUDF77L6BP4IMIMR6KRZ2FTZJ6BG5TMAZGIW75PHQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228904886 (189h52m27.056613283s) [noise=true , address=89.149.201.147:4024 , piece_id=7MCMBHNW4L7JDB4ATRHAM3GWVFAYH34LWFNX5O7C6EXIB34BNWRA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228954088 (189h52m27.056662491s) [seg_pos={0 0}, upload=197386]: Segment reached optimal threshold of 26 pieces. Dec 13 13:20:26.229843002 (189h52m27.057551404s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: read tcp 159.223.143.39:53228->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.229857857 (189h52m27.05756626s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230208353 (189h52m27.057916754s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7): protocol: read tcp 159.223.143.39:45318->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.230219436 (189h52m27.057927837s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230249796 (189h52m27.057958176s) [seg_pos={0 0}, upload=197386]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 13 13:20:26.230536147 (189h52m27.05824454s) [upload=197386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 13 13:20:26.230543508 (189h52m27.058251899s) [upload=197386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 13 13:20:26.379840216 (189h52m27.207548608s) [upload=197386]: Done uploading segments... Dec 13 13:20:26.379861196 (189h52m27.207569587s) [upload=197386]: Upload finished. err: <nil> Duration: 35.627039703s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 53.217620913s trace: https://jaeger.dev.storj.io/trace/12dbee1725a89af6 FAIL (2025-12-13 13:24:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=197386]: Uploading piece... Dec 13 13:20:22.157523997 (189h52m22.985232392s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157563526 (189h52m22.985271946s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157782226 (189h52m22.985490616s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158220136 (189h52m22.98592853s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158561593 (189h52m22.986269971s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158833133 (189h52m22.986541524s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159110305 (189h52m22.986818699s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159379461 (189h52m22.987087838s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159627778 (189h52m22.987336156s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.588165740 (189h52m23.41587413s) [upload=197386]: Next returned nil segment Dec 13 13:20:22.588188840 (189h52m23.415897231s) [upload=197386]: Waiting for error group managing segments... Dec 13 13:20:22.787355936 (189h52m23.615064332s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.793561088 (189h52m23.621269481s) [noise=true , address=185.167.204.205:4013 , piece_id=QNJLAFTMBQPQF7IBMT6EKMBEL5KX7VHU3L4L3ALPXYNUV2MZ43GA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.801170609 (189h52m23.628879002s) [noise=true , address=185.167.204.206:4068 , piece_id=P2FTDV3XW34Z6M73CWIURQCCQ6H5NXDNJWKBVN3GRQTDV65QLP5A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.812199494 (189h52m23.63990789s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813259194 (189h52m23.640967588s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813831001 (189h52m23.641539395s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814310957 (189h52m23.642019347s) [noise=true , address=185.167.204.205:4001 , piece_id=WXEZMLB4ZM2FFOB4JMCIF2X6FTB72I3EWWRTDLIZ6SXCMUTS5HGQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814961478 (189h52m23.64266987s) [noise=true , address=185.167.204.206:4046 , piece_id=KLHWCCQLK3CAOIMW57WAFZHNMECYLGA4VRZE3LXRUJRQCKAGGBKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.815469120 (189h52m23.643177512s) [noise=true , address=185.167.204.206:4054 , piece_id=ZPZSVEBAXMRI7732XKIHUGMU2WGGCNKE34U2FZ3NHJLKCMYBIUXQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.817780190 (189h52m23.645488584s) [noise=true , address=185.167.204.206:4045 , piece_id=VQ37X765RRBAVPI4HIAPJET7VRCI2FE7FRHB45II6ETFYDYMKSTQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818186706 (189h52m23.645895101s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818345893 (189h52m23.646054284s) [noise=true , address=185.167.204.205:4014 , piece_id=4U6VXOGIXKAV7NJBQBD2V6ZVWAUIT557OP7AUG332UWVS5LZFSPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.844792090 (189h52m23.672516847s) [noise=true , address=185.167.204.206:4051 , piece_id=A4VZAGPIVLUVOVOIGGIK7CEDRPUSV34XMUL5TFL5KFJNICAWSJ4Q, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.354540799 (189h52m25.182249185s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.369904036 (189h52m25.197612417s) [noise=true , address=81.171.14.70:4039 , piece_id=YV7FP35KY6TXYDKBEOV6RCHCSJWR4TQCFEB43GK3AZP4C776BCZQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.378758619 (189h52m25.206467012s) [noise=true , address=81.171.14.70:4047 , piece_id=EQFLTLGTZBKQ47MWFLSBCEHDKAWMZSGJHZPBAN67EANSP2LR7HYQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.384761037 (189h52m25.212469431s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.495107931 (189h52m25.322816323s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.833114410 (189h52m25.660822801s) [noise=true , address=89.149.201.147:4009 , piece_id=SAES2PXHWVMC22L3DKLBYHAI3ZHKYKGYOULL4ZZ2AB2U53B5URVA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.076006180 (189h52m25.903714572s) [noise=true , address=89.149.201.147:4008 , piece_id=HL2EBKXU5RWRNZP7D3MUOTRJK2YJTADFQS6C4IF7CGEUF3PLOIMQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.109256616 (189h52m25.936964999s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.121296573 (189h52m25.949004965s) [noise=true , address=81.171.14.70:4034 , piece_id=XZ72ODCL37AYLPMMFTCT4NJROVRCLTEM2XN6SQVG3G5NTZRJDWDA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.144861937 (189h52m25.972570318s) [noise=true , address=89.149.201.147:4012 , piece_id=5LDACT5ZGR54I4LNLZQHNXUY2VZQ5XIWI5AYUM5CZHCOOQIBGPRQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.859441967 (189h52m26.687150358s) [noise=true , address=81.171.14.70:4026 , piece_id=R7B2VE64HYLI7A2XVWDFTIIWAVLT6I6E75ZCAMTDH77R7BW2Q7IQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.932179545 (189h52m26.759887944s) [noise=true , address=89.149.201.147:4001 , piece_id=GDNVHYCJJDXLUDF77L6BP4IMIMR6KRZ2FTZJ6BG5TMAZGIW75PHQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228904886 (189h52m27.056613283s) [noise=true , address=89.149.201.147:4024 , piece_id=7MCMBHNW4L7JDB4ATRHAM3GWVFAYH34LWFNX5O7C6EXIB34BNWRA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228954088 (189h52m27.056662491s) [seg_pos={0 0}, upload=197386]: Segment reached optimal threshold of 26 pieces. Dec 13 13:20:26.229843002 (189h52m27.057551404s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: read tcp 159.223.143.39:53228->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.229857857 (189h52m27.05756626s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230208353 (189h52m27.057916754s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7): protocol: read tcp 159.223.143.39:45318->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.230219436 (189h52m27.057927837s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230249796 (189h52m27.057958176s) [seg_pos={0 0}, upload=197386]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 13 13:20:26.230536147 (189h52m27.05824454s) [upload=197386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 13 13:20:26.230543508 (189h52m27.058251899s) [upload=197386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 13 13:20:26.379840216 (189h52m27.207548608s) [upload=197386]: Done uploading segments... Dec 13 13:20:26.379861196 (189h52m27.207569587s) [upload=197386]: Upload finished. err: <nil> Duration: 53.21763865s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 50.97609898s trace: https://jaeger.dev.storj.io/trace/12dbee1725a89af6 FAIL (2025-12-13 13:23:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=197386]: Uploading piece... Dec 13 13:20:22.157523997 (189h52m22.985232392s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157563526 (189h52m22.985271946s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157782226 (189h52m22.985490616s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158220136 (189h52m22.98592853s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158561593 (189h52m22.986269971s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158833133 (189h52m22.986541524s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159110305 (189h52m22.986818699s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159379461 (189h52m22.987087838s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159627778 (189h52m22.987336156s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.588165740 (189h52m23.41587413s) [upload=197386]: Next returned nil segment Dec 13 13:20:22.588188840 (189h52m23.415897231s) [upload=197386]: Waiting for error group managing segments... Dec 13 13:20:22.787355936 (189h52m23.615064332s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.793561088 (189h52m23.621269481s) [noise=true , address=185.167.204.205:4013 , piece_id=QNJLAFTMBQPQF7IBMT6EKMBEL5KX7VHU3L4L3ALPXYNUV2MZ43GA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.801170609 (189h52m23.628879002s) [noise=true , address=185.167.204.206:4068 , piece_id=P2FTDV3XW34Z6M73CWIURQCCQ6H5NXDNJWKBVN3GRQTDV65QLP5A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.812199494 (189h52m23.63990789s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813259194 (189h52m23.640967588s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813831001 (189h52m23.641539395s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814310957 (189h52m23.642019347s) [noise=true , address=185.167.204.205:4001 , piece_id=WXEZMLB4ZM2FFOB4JMCIF2X6FTB72I3EWWRTDLIZ6SXCMUTS5HGQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814961478 (189h52m23.64266987s) [noise=true , address=185.167.204.206:4046 , piece_id=KLHWCCQLK3CAOIMW57WAFZHNMECYLGA4VRZE3LXRUJRQCKAGGBKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.815469120 (189h52m23.643177512s) [noise=true , address=185.167.204.206:4054 , piece_id=ZPZSVEBAXMRI7732XKIHUGMU2WGGCNKE34U2FZ3NHJLKCMYBIUXQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.817780190 (189h52m23.645488584s) [noise=true , address=185.167.204.206:4045 , piece_id=VQ37X765RRBAVPI4HIAPJET7VRCI2FE7FRHB45II6ETFYDYMKSTQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818186706 (189h52m23.645895101s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818345893 (189h52m23.646054284s) [noise=true , address=185.167.204.205:4014 , piece_id=4U6VXOGIXKAV7NJBQBD2V6ZVWAUIT557OP7AUG332UWVS5LZFSPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.844792090 (189h52m23.672516847s) [noise=true , address=185.167.204.206:4051 , piece_id=A4VZAGPIVLUVOVOIGGIK7CEDRPUSV34XMUL5TFL5KFJNICAWSJ4Q, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.354540799 (189h52m25.182249185s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.369904036 (189h52m25.197612417s) [noise=true , address=81.171.14.70:4039 , piece_id=YV7FP35KY6TXYDKBEOV6RCHCSJWR4TQCFEB43GK3AZP4C776BCZQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.378758619 (189h52m25.206467012s) [noise=true , address=81.171.14.70:4047 , piece_id=EQFLTLGTZBKQ47MWFLSBCEHDKAWMZSGJHZPBAN67EANSP2LR7HYQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.384761037 (189h52m25.212469431s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.495107931 (189h52m25.322816323s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.833114410 (189h52m25.660822801s) [noise=true , address=89.149.201.147:4009 , piece_id=SAES2PXHWVMC22L3DKLBYHAI3ZHKYKGYOULL4ZZ2AB2U53B5URVA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.076006180 (189h52m25.903714572s) [noise=true , address=89.149.201.147:4008 , piece_id=HL2EBKXU5RWRNZP7D3MUOTRJK2YJTADFQS6C4IF7CGEUF3PLOIMQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.109256616 (189h52m25.936964999s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.121296573 (189h52m25.949004965s) [noise=true , address=81.171.14.70:4034 , piece_id=XZ72ODCL37AYLPMMFTCT4NJROVRCLTEM2XN6SQVG3G5NTZRJDWDA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.144861937 (189h52m25.972570318s) [noise=true , address=89.149.201.147:4012 , piece_id=5LDACT5ZGR54I4LNLZQHNXUY2VZQ5XIWI5AYUM5CZHCOOQIBGPRQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.859441967 (189h52m26.687150358s) [noise=true , address=81.171.14.70:4026 , piece_id=R7B2VE64HYLI7A2XVWDFTIIWAVLT6I6E75ZCAMTDH77R7BW2Q7IQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.932179545 (189h52m26.759887944s) [noise=true , address=89.149.201.147:4001 , piece_id=GDNVHYCJJDXLUDF77L6BP4IMIMR6KRZ2FTZJ6BG5TMAZGIW75PHQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228904886 (189h52m27.056613283s) [noise=true , address=89.149.201.147:4024 , piece_id=7MCMBHNW4L7JDB4ATRHAM3GWVFAYH34LWFNX5O7C6EXIB34BNWRA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228954088 (189h52m27.056662491s) [seg_pos={0 0}, upload=197386]: Segment reached optimal threshold of 26 pieces. Dec 13 13:20:26.229843002 (189h52m27.057551404s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: read tcp 159.223.143.39:53228->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.229857857 (189h52m27.05756626s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230208353 (189h52m27.057916754s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7): protocol: read tcp 159.223.143.39:45318->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.230219436 (189h52m27.057927837s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230249796 (189h52m27.057958176s) [seg_pos={0 0}, upload=197386]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 13 13:20:26.230536147 (189h52m27.05824454s) [upload=197386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 13 13:20:26.230543508 (189h52m27.058251899s) [upload=197386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 13 13:20:26.379840216 (189h52m27.207548608s) [upload=197386]: Done uploading segments... Dec 13 13:20:26.379861196 (189h52m27.207569587s) [upload=197386]: Upload finished. err: <nil> Duration: 50.976125983s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 38.390104315s trace: https://jaeger.dev.storj.io/trace/12dbee1725a89af6 FAIL (2025-12-13 13:22:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=197386]: Uploading piece... Dec 13 13:20:22.157523997 (189h52m22.985232392s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157563526 (189h52m22.985271946s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157782226 (189h52m22.985490616s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158220136 (189h52m22.98592853s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158561593 (189h52m22.986269971s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158833133 (189h52m22.986541524s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159110305 (189h52m22.986818699s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159379461 (189h52m22.987087838s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159627778 (189h52m22.987336156s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.588165740 (189h52m23.41587413s) [upload=197386]: Next returned nil segment Dec 13 13:20:22.588188840 (189h52m23.415897231s) [upload=197386]: Waiting for error group managing segments... Dec 13 13:20:22.787355936 (189h52m23.615064332s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.793561088 (189h52m23.621269481s) [noise=true , address=185.167.204.205:4013 , piece_id=QNJLAFTMBQPQF7IBMT6EKMBEL5KX7VHU3L4L3ALPXYNUV2MZ43GA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.801170609 (189h52m23.628879002s) [noise=true , address=185.167.204.206:4068 , piece_id=P2FTDV3XW34Z6M73CWIURQCCQ6H5NXDNJWKBVN3GRQTDV65QLP5A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.812199494 (189h52m23.63990789s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813259194 (189h52m23.640967588s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813831001 (189h52m23.641539395s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814310957 (189h52m23.642019347s) [noise=true , address=185.167.204.205:4001 , piece_id=WXEZMLB4ZM2FFOB4JMCIF2X6FTB72I3EWWRTDLIZ6SXCMUTS5HGQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814961478 (189h52m23.64266987s) [noise=true , address=185.167.204.206:4046 , piece_id=KLHWCCQLK3CAOIMW57WAFZHNMECYLGA4VRZE3LXRUJRQCKAGGBKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.815469120 (189h52m23.643177512s) [noise=true , address=185.167.204.206:4054 , piece_id=ZPZSVEBAXMRI7732XKIHUGMU2WGGCNKE34U2FZ3NHJLKCMYBIUXQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.817780190 (189h52m23.645488584s) [noise=true , address=185.167.204.206:4045 , piece_id=VQ37X765RRBAVPI4HIAPJET7VRCI2FE7FRHB45II6ETFYDYMKSTQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818186706 (189h52m23.645895101s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818345893 (189h52m23.646054284s) [noise=true , address=185.167.204.205:4014 , piece_id=4U6VXOGIXKAV7NJBQBD2V6ZVWAUIT557OP7AUG332UWVS5LZFSPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.844792090 (189h52m23.672516847s) [noise=true , address=185.167.204.206:4051 , piece_id=A4VZAGPIVLUVOVOIGGIK7CEDRPUSV34XMUL5TFL5KFJNICAWSJ4Q, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.354540799 (189h52m25.182249185s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.369904036 (189h52m25.197612417s) [noise=true , address=81.171.14.70:4039 , piece_id=YV7FP35KY6TXYDKBEOV6RCHCSJWR4TQCFEB43GK3AZP4C776BCZQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.378758619 (189h52m25.206467012s) [noise=true , address=81.171.14.70:4047 , piece_id=EQFLTLGTZBKQ47MWFLSBCEHDKAWMZSGJHZPBAN67EANSP2LR7HYQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.384761037 (189h52m25.212469431s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.495107931 (189h52m25.322816323s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.833114410 (189h52m25.660822801s) [noise=true , address=89.149.201.147:4009 , piece_id=SAES2PXHWVMC22L3DKLBYHAI3ZHKYKGYOULL4ZZ2AB2U53B5URVA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.076006180 (189h52m25.903714572s) [noise=true , address=89.149.201.147:4008 , piece_id=HL2EBKXU5RWRNZP7D3MUOTRJK2YJTADFQS6C4IF7CGEUF3PLOIMQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.109256616 (189h52m25.936964999s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.121296573 (189h52m25.949004965s) [noise=true , address=81.171.14.70:4034 , piece_id=XZ72ODCL37AYLPMMFTCT4NJROVRCLTEM2XN6SQVG3G5NTZRJDWDA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.144861937 (189h52m25.972570318s) [noise=true , address=89.149.201.147:4012 , piece_id=5LDACT5ZGR54I4LNLZQHNXUY2VZQ5XIWI5AYUM5CZHCOOQIBGPRQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.859441967 (189h52m26.687150358s) [noise=true , address=81.171.14.70:4026 , piece_id=R7B2VE64HYLI7A2XVWDFTIIWAVLT6I6E75ZCAMTDH77R7BW2Q7IQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.932179545 (189h52m26.759887944s) [noise=true , address=89.149.201.147:4001 , piece_id=GDNVHYCJJDXLUDF77L6BP4IMIMR6KRZ2FTZJ6BG5TMAZGIW75PHQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228904886 (189h52m27.056613283s) [noise=true , address=89.149.201.147:4024 , piece_id=7MCMBHNW4L7JDB4ATRHAM3GWVFAYH34LWFNX5O7C6EXIB34BNWRA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228954088 (189h52m27.056662491s) [seg_pos={0 0}, upload=197386]: Segment reached optimal threshold of 26 pieces. Dec 13 13:20:26.229843002 (189h52m27.057551404s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: read tcp 159.223.143.39:53228->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.229857857 (189h52m27.05756626s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230208353 (189h52m27.057916754s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7): protocol: read tcp 159.223.143.39:45318->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.230219436 (189h52m27.057927837s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230249796 (189h52m27.057958176s) [seg_pos={0 0}, upload=197386]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 13 13:20:26.230536147 (189h52m27.05824454s) [upload=197386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 13 13:20:26.230543508 (189h52m27.058251899s) [upload=197386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 13 13:20:26.379840216 (189h52m27.207548608s) [upload=197386]: Done uploading segments... Dec 13 13:20:26.379861196 (189h52m27.207569587s) [upload=197386]: Upload finished. err: <nil> Duration: 38.390127695s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.003233264s trace: https://jaeger.dev.storj.io/trace/12dbee1725a89af6 FAIL (2025-12-13 13:21:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=197386]: Uploading piece... Dec 13 13:20:22.157523997 (189h52m22.985232392s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157563526 (189h52m22.985271946s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.157782226 (189h52m22.985490616s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158220136 (189h52m22.98592853s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158561593 (189h52m22.986269971s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.158833133 (189h52m22.986541524s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159110305 (189h52m22.986818699s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159379461 (189h52m22.987087838s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.159627778 (189h52m22.987336156s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Uploading piece... Dec 13 13:20:22.588165740 (189h52m23.41587413s) [upload=197386]: Next returned nil segment Dec 13 13:20:22.588188840 (189h52m23.415897231s) [upload=197386]: Waiting for error group managing segments... Dec 13 13:20:22.787355936 (189h52m23.615064332s) [noise=true , address=185.167.204.205:4005 , piece_id=B5NHOBIEY4KGWHIUBAP2N6FWVIMFBSWUS4DDWQVFMKSSSB654UPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.793561088 (189h52m23.621269481s) [noise=true , address=185.167.204.205:4013 , piece_id=QNJLAFTMBQPQF7IBMT6EKMBEL5KX7VHU3L4L3ALPXYNUV2MZ43GA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.801170609 (189h52m23.628879002s) [noise=true , address=185.167.204.206:4068 , piece_id=P2FTDV3XW34Z6M73CWIURQCCQ6H5NXDNJWKBVN3GRQTDV65QLP5A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.812199494 (189h52m23.63990789s) [noise=true , address=185.167.204.206:4048 , piece_id=MOJVQUBRM36ENNJVV7EYAXQEZQIXN7S5UBP3SBHERO7LVFDZ4HIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813259194 (189h52m23.640967588s) [noise=true , address=185.167.204.205:4006 , piece_id=DFHXNKDAWUBPQ4KGMDTUQ76MJIDUAX5U7OQIPNXJYRELBIVXYVVQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.813831001 (189h52m23.641539395s) [noise=true , address=185.167.204.206:4056 , piece_id=TYQW5XG5HOLULH5R7UGMEDCTMLXOWEA3A6WEBN6EVI5UNUZ5HH2A, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814310957 (189h52m23.642019347s) [noise=true , address=185.167.204.205:4001 , piece_id=WXEZMLB4ZM2FFOB4JMCIF2X6FTB72I3EWWRTDLIZ6SXCMUTS5HGQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.814961478 (189h52m23.64266987s) [noise=true , address=185.167.204.206:4046 , piece_id=KLHWCCQLK3CAOIMW57WAFZHNMECYLGA4VRZE3LXRUJRQCKAGGBKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.815469120 (189h52m23.643177512s) [noise=true , address=185.167.204.206:4054 , piece_id=ZPZSVEBAXMRI7732XKIHUGMU2WGGCNKE34U2FZ3NHJLKCMYBIUXQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.817780190 (189h52m23.645488584s) [noise=true , address=185.167.204.206:4045 , piece_id=VQ37X765RRBAVPI4HIAPJET7VRCI2FE7FRHB45II6ETFYDYMKSTQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818186706 (189h52m23.645895101s) [noise=true , address=185.167.204.206:4038 , piece_id=MKQFEVYRSI4HGUMDZZGQEIMROWZAQDFWQYJ3TPCZE56ROFBY6GBQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.818345893 (189h52m23.646054284s) [noise=true , address=185.167.204.205:4014 , piece_id=4U6VXOGIXKAV7NJBQBD2V6ZVWAUIT557OP7AUG332UWVS5LZFSPA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:22.844792090 (189h52m23.672516847s) [noise=true , address=185.167.204.206:4051 , piece_id=A4VZAGPIVLUVOVOIGGIK7CEDRPUSV34XMUL5TFL5KFJNICAWSJ4Q, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.354540799 (189h52m25.182249185s) [noise=true , address=81.171.14.70:4033 , piece_id=5NKXLV6JVGJJJYMT7IUC3GR5LRRXGS5F6Y6IEMX24DIQT7FTXXIA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.369904036 (189h52m25.197612417s) [noise=true , address=81.171.14.70:4039 , piece_id=YV7FP35KY6TXYDKBEOV6RCHCSJWR4TQCFEB43GK3AZP4C776BCZQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.378758619 (189h52m25.206467012s) [noise=true , address=81.171.14.70:4047 , piece_id=EQFLTLGTZBKQ47MWFLSBCEHDKAWMZSGJHZPBAN67EANSP2LR7HYQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.384761037 (189h52m25.212469431s) [noise=true , address=89.149.201.147:4006 , piece_id=LE3FPTMI6GWHTCCWL5H4MZPEGLHCTBWHWCACVUOIQWCU62JYIRCQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.495107931 (189h52m25.322816323s) [noise=true , address=81.171.14.70:4029 , piece_id=EOO6MF4FIN6OLOXXBS7GDFF4IMK2YYH75CFI4OGNNOYRCAHCSAKQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:24.833114410 (189h52m25.660822801s) [noise=true , address=89.149.201.147:4009 , piece_id=SAES2PXHWVMC22L3DKLBYHAI3ZHKYKGYOULL4ZZ2AB2U53B5URVA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.076006180 (189h52m25.903714572s) [noise=true , address=89.149.201.147:4008 , piece_id=HL2EBKXU5RWRNZP7D3MUOTRJK2YJTADFQS6C4IF7CGEUF3PLOIMQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.109256616 (189h52m25.936964999s) [noise=true , address=89.149.201.147:4022 , piece_id=3VH5FONJYYDQQ4AK74JQTD26TTLRZPTRCZH7WLHXX5Q7Y2OXXIZA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.121296573 (189h52m25.949004965s) [noise=true , address=81.171.14.70:4034 , piece_id=XZ72ODCL37AYLPMMFTCT4NJROVRCLTEM2XN6SQVG3G5NTZRJDWDA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.144861937 (189h52m25.972570318s) [noise=true , address=89.149.201.147:4012 , piece_id=5LDACT5ZGR54I4LNLZQHNXUY2VZQ5XIWI5AYUM5CZHCOOQIBGPRQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.859441967 (189h52m26.687150358s) [noise=true , address=81.171.14.70:4026 , piece_id=R7B2VE64HYLI7A2XVWDFTIIWAVLT6I6E75ZCAMTDH77R7BW2Q7IQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:25.932179545 (189h52m26.759887944s) [noise=true , address=89.149.201.147:4001 , piece_id=GDNVHYCJJDXLUDF77L6BP4IMIMR6KRZ2FTZJ6BG5TMAZGIW75PHQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228904886 (189h52m27.056613283s) [noise=true , address=89.149.201.147:4024 , piece_id=7MCMBHNW4L7JDB4ATRHAM3GWVFAYH34LWFNX5O7C6EXIB34BNWRA, seg_pos={0 0}, upload=197386]: Done uploading piece. err: <nil> Dec 13 13:20:26.228954088 (189h52m27.056662491s) [seg_pos={0 0}, upload=197386]: Segment reached optimal threshold of 26 pieces. Dec 13 13:20:26.229843002 (189h52m27.057551404s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: read tcp 159.223.143.39:53228->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.229857857 (189h52m27.05756626s) [noise=true , address=81.171.14.70:4027 , piece_id=ANUN37MZAFWJ3FTQDU6KXRNZOLN2D62GA6AL65ENUEXX2XJEE3GQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230208353 (189h52m27.057916754s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7): protocol: read tcp 159.223.143.39:45318->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 13 13:20:26.230219436 (189h52m27.057927837s) [noise=true , address=81.171.14.70:4046 , piece_id=T2PFYRKNF7IFA2275HKTEP6WBDBZKL56BZ26H7IHDNMWWRX56JJQ, seg_pos={0 0}, upload=197386]: Long tail cancellation due to optimal threshold Dec 13 13:20:26.230249796 (189h52m27.057958176s) [seg_pos={0 0}, upload=197386]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 13 13:20:26.230536147 (189h52m27.05824454s) [upload=197386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 13 13:20:26.230543508 (189h52m27.058251899s) [upload=197386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 13 13:20:26.379840216 (189h52m27.207548608s) [upload=197386]: Done uploading segments... Dec 13 13:20:26.379861196 (189h52m27.207569587s) [upload=197386]: Upload finished. err: <nil> Duration: 1m0.003256016s core uplink eu-select-1 eu1-placement-15
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 39.576151ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace: https://jaeger.dev.storj.io/trace/66bd83328d99d73b FAIL (2025-12-13 05:31:16 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 159.223.143.39:43650->45.38.24.2:443: read: connection reset by peer Duration: 39.601714ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 41.047861ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace: https://jaeger.dev.storj.io/trace/66bd83328d99d73b FAIL (2025-12-13 05:30:43 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 159.223.143.39:43650->45.38.24.2:443: read: connection reset by peer Duration: 41.094647ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 46.202176ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace: https://jaeger.dev.storj.io/trace/66bd83328d99d73b FAIL (2025-12-13 05:29:28 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 159.223.143.39:43650->45.38.24.2:443: read: connection reset by peer Duration: 46.319885ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 40.840378ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace: https://jaeger.dev.storj.io/trace/66bd83328d99d73b FAIL (2025-12-13 05:29:04 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 159.223.143.39:43650->45.38.24.2:443: read: connection reset by peer Duration: 40.860495ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 41.75253ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace: https://jaeger.dev.storj.io/trace/66bd83328d99d73b FAIL (2025-12-13 05:27:59 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 159.223.143.39:43650->45.38.24.2:443: read: connection reset by peer Duration: 41.805885ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 45.805462ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace: https://jaeger.dev.storj.io/trace/66bd83328d99d73b FAIL (2025-12-13 05:26:47 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 159.223.143.39:43650->45.38.24.2:443: read: connection reset by peer Duration: 45.829707ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 42.910029ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace: https://jaeger.dev.storj.io/trace/66bd83328d99d73b FAIL (2025-12-13 05:25:57 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 159.223.143.39:43650->45.38.24.2:443: read: connection reset by peer Duration: 42.933118ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 41.963121ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace: https://jaeger.dev.storj.io/trace/66bd83328d99d73b FAIL (2025-12-13 05:24:45 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 159.223.143.39:43650->45.38.24.2:443: read: connection reset by peer Duration: 41.99165ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 39.420625ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace: https://jaeger.dev.storj.io/trace/66bd83328d99d73b FAIL (2025-12-13 05:23:57 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 159.223.143.39:43650->45.38.24.2:443: read: connection reset by peer Duration: 39.44623ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 39.582522ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace: https://jaeger.dev.storj.io/trace/66bd83328d99d73b FAIL (2025-12-13 05:23:22 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 159.223.143.39:43650->45.38.24.2:443: read: connection reset by peer Duration: 39.609223ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 54.007864ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace: https://jaeger.dev.storj.io/trace/66bd83328d99d73b FAIL (2025-12-13 05:23:00 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 159.223.143.39:43650->45.38.24.2:443: read: connection reset by peer Duration: 54.032235ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 42.535021ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace: https://jaeger.dev.storj.io/trace/66bd83328d99d73b FAIL (2025-12-13 05:22:01 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 159.223.143.39:43650->45.38.24.2:443: read: connection reset by peer Duration: 42.628474ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 38.205944ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace: https://jaeger.dev.storj.io/trace/66bd83328d99d73b FAIL (2025-12-13 05:21:18 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 159.223.143.39:43650->45.38.24.2:443: read: connection reset by peer Duration: 38.236885ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 58.272192ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace: https://jaeger.dev.storj.io/trace/66bd83328d99d73b FAIL (2025-12-13 05:20:55 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 159.223.143.39:43650->45.38.24.2:443: read: connection reset by peer Duration: 58.366347ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 41.223082ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace: https://jaeger.dev.storj.io/trace/66bd83328d99d73b FAIL (2025-12-13 05:19:48 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 159.223.143.39:43650->45.38.24.2:443: read: connection reset by peer Duration: 41.249626ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 70.580639ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace: https://jaeger.dev.storj.io/trace/66bd83328d99d73b FAIL (2025-12-13 05:19:00 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 159.223.143.39:43650->45.38.24.2:443: read: connection reset by peer Duration: 70.671005ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 38.857685ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace: https://jaeger.dev.storj.io/trace/66bd83328d99d73b FAIL (2025-12-13 05:18:23 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 159.223.143.39:43650->45.38.24.2:443: read: connection reset by peer Duration: 38.883481ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 76.997592ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace: https://jaeger.dev.storj.io/trace/66bd83328d99d73b FAIL (2025-12-13 05:17:27 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 159.223.143.39:43650->45.38.24.2:443: read: connection reset by peer Duration: 77.020328ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 44.983455ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace: https://jaeger.dev.storj.io/trace/66bd83328d99d73b FAIL (2025-12-13 05:16:48 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 159.223.143.39:43650->45.38.24.2:443: read: connection reset by peer Duration: 45.027093ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 40.699336ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace: https://jaeger.dev.storj.io/trace/66bd83328d99d73b FAIL (2025-12-13 05:15:43 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 159.223.143.39:43650->45.38.24.2:443: read: connection reset by peer Duration: 40.724079ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 45.718058ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace: https://jaeger.dev.storj.io/trace/66bd83328d99d73b FAIL (2025-12-13 05:14:55 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 159.223.143.39:43650->45.38.24.2:443: read: connection reset by peer Duration: 45.739872ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 44.424136ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace: https://jaeger.dev.storj.io/trace/66bd83328d99d73b FAIL (2025-12-13 05:13:51 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 159.223.143.39:43650->45.38.24.2:443: read: connection reset by peer Duration: 44.450162ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 41.982149ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace: https://jaeger.dev.storj.io/trace/66bd83328d99d73b FAIL (2025-12-13 05:12:37 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 159.223.143.39:43650->45.38.24.2:443: read: connection reset by peer Duration: 42.006796ms edge gateway vhost us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)