Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 301.545233ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:30:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 301.561641ms core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.393743935s trace: https://jaeger.dev.storj.io/trace/2978a773c720f2e4 FAIL (2025-11-20 17:30:38 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== load=232637]: Uploading piece... Nov 20 17:25:10.582877896 (223h38m35.350561328s) [noise=true , address=81.171.14.70:4029 , piece_id=XJF5EGWPXCL3PXV6WXEL46JAHMHZQGGR4AU5IOUNQLBG4C3BGQUA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.584580409 (223h38m35.352263843s) [noise=true , address=185.167.204.205:4026 , piece_id=YVFA5OFDVPAN3Q7GP7NIO6MC2N47G2MEWLJ7RVJEIP7UBHBYD5NA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.586378791 (223h38m35.354062219s) [noise=true , address=185.167.204.205:4036 , piece_id=LZSPRFBBIJPCW3IWHNW3SNKX4YTW7RCXTOOLE5SBGR2Z4WJHVCZA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.588641711 (223h38m35.356325142s) [noise=true , address=185.167.204.205:4015 , piece_id=ZC3TZGH2N7NHRNMFGG2SVOS5UGTD4CY4OCLCQL2EEIN6RHEMQKIQ, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.591403897 (223h38m35.359087325s) [noise=true , address=185.167.204.205:4034 , piece_id=MPKSGXYASP3YGAXWHL4PLGA7PJX56ELPS6NPJQIKLQN3GOYAUFDA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.593348617 (223h38m35.361032051s) [noise=true , address=81.171.14.70:4031 , piece_id=SHBKH5K4C7T7D7XMAAD6FDA36VCEJXQILDGEVNJZLTMD6EXLBL5A, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.595567737 (223h38m35.363251162s) [noise=true , address=185.167.204.205:4002 , piece_id=27KMJOJSSJTIHEO5ZKY66GIG66P5QAESU4HHHLTI6KV2WZCFRWYQ, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.597504768 (223h38m35.365188202s) [noise=true , address=81.171.14.70:4028 , piece_id=HEZAKOJDOQEBIFDRGYIXSCUWLPKUBTA6MRKQPR6665ZQXEPVIGQA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.599544427 (223h38m35.367227861s) [noise=true , address=81.171.14.70:4038 , piece_id=GCUUKZQY36F6XWCXJGHP6SSK5OKAH47HLPNBC76GVNNQID4DVLDA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.600434395 (223h38m35.36811783s) [noise=true , address=185.167.204.206:4065 , piece_id=UHVHSM2N5H73YGCYEJ26JWO6IZZ76JTTYWM5ZZZZ2HMERRK7VDWA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.992513100 (223h38m35.760196525s) [upload=232637]: Next returned nil segment Nov 20 17:25:10.992541848 (223h38m35.760225277s) [upload=232637]: Waiting for error group managing segments... Nov 20 17:25:11.164544942 (223h38m35.932228366s) [noise=true , address=89.149.201.147:4022 , piece_id=4NQ4J3UJAABVO3SFMRORPXBTUGYXZMJNUWWTAKFIH3ZD5EVF3UXQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.172767990 (223h38m35.940451418s) [noise=true , address=81.171.14.70:4031 , piece_id=SHBKH5K4C7T7D7XMAAD6FDA36VCEJXQILDGEVNJZLTMD6EXLBL5A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.182086404 (223h38m35.949769833s) [noise=true , address=185.167.204.205:4015 , piece_id=ZC3TZGH2N7NHRNMFGG2SVOS5UGTD4CY4OCLCQL2EEIN6RHEMQKIQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.182810476 (223h38m35.950493902s) [noise=true , address=81.171.14.70:4043 , piece_id=6474UQDNSX4UVDXJWZKSDBB5T47HGTYJZMJNTFK3PEQBR2KAY7RA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.183742078 (223h38m35.951425504s) [noise=true , address=185.167.204.206:4053 , piece_id=AIF5XPKJPTGUA7ENKUUTPPOKQCECEKQPPGEMAUOT5QAISCVMO22Q, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.184529255 (223h38m35.952212684s) [noise=true , address=81.171.14.70:4044 , piece_id=RICD4ZMMWVGLS6UTFKKZFUNKE2LI35QQ6VF4FYRMLDUMQDOQCCXA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.185123500 (223h38m35.952806926s) [noise=true , address=185.167.204.206:4066 , piece_id=H52XDJBLWKNNCE25VSBRNUH6BMW3JPBOFGQSEKF4CI3HBKCQSJZQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.186964052 (223h38m35.954647478s) [noise=true , address=185.167.204.205:4028 , piece_id=FVBJ6WVHKSN43OVNMH66KPBREGS2EWGD2EWCVJQX2WVMI27MPC7A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.191410051 (223h38m35.959093477s) [noise=true , address=185.167.204.206:4054 , piece_id=LCBZUECG7VIQOE3X4VT6AT4HEQEEOTHEFBPSRXG7NUJC4PCGQ72Q, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.194616300 (223h38m35.962299725s) [noise=true , address=89.149.201.147:4001 , piece_id=7YTUOPW4BA4KDNJAXDHNUENBDIXNGZZW6576XQD5D3ISQXKBTYTQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.195124666 (223h38m35.962808091s) [noise=true , address=81.171.14.70:4027 , piece_id=OOS4I4CDV2TN42MWMHZBRCQWXGYXLHJTTUONDMTAHSQJKKPROYJA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.196224298 (223h38m35.963907724s) [noise=true , address=89.149.201.147:4005 , piece_id=U64XSFTL2TDVWNNOUQ6ULUGBPNU5XGQ6SI3HU7ZP6PPLZ7PZVRQA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.198703124 (223h38m35.966386601s) [noise=true , address=185.167.204.206:4047 , piece_id=YZFK5OW7TWWIJ6FLA2XMAU3YURPKT2U5FGZYH6YLRNHUUDIV5N7A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.201965022 (223h38m35.969648447s) [noise=true , address=185.167.204.205:4034 , piece_id=MPKSGXYASP3YGAXWHL4PLGA7PJX56ELPS6NPJQIKLQN3GOYAUFDA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.202483607 (223h38m35.970167033s) [noise=true , address=185.167.204.205:4002 , piece_id=27KMJOJSSJTIHEO5ZKY66GIG66P5QAESU4HHHLTI6KV2WZCFRWYQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.211202242 (223h38m35.978885672s) [noise=true , address=185.167.204.205:4036 , piece_id=LZSPRFBBIJPCW3IWHNW3SNKX4YTW7RCXTOOLE5SBGR2Z4WJHVCZA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.216399102 (223h38m35.984082529s) [noise=true , address=185.167.204.206:4046 , piece_id=VY3VEVR5VIHEH3MWJKDYS2UM62EP7ELZU3JI346NDESEORJIPZDQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.226445839 (223h38m35.994129265s) [noise=true , address=185.167.204.205:4026 , piece_id=YVFA5OFDVPAN3Q7GP7NIO6MC2N47G2MEWLJ7RVJEIP7UBHBYD5NA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.240411560 (223h38m36.008094989s) [noise=true , address=185.167.204.206:4065 , piece_id=UHVHSM2N5H73YGCYEJ26JWO6IZZ76JTTYWM5ZZZZ2HMERRK7VDWA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.240958840 (223h38m36.008642266s) [noise=true , address=81.171.14.70:4047 , piece_id=D46YVEDXW2RT6OUYO73PHKREJF55SOCRWX6AMJ6RUWUC7G67426A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.241565965 (223h38m36.009249392s) [noise=true , address=81.171.14.70:4036 , piece_id=RNDJKD3UBSBBCUSY6AMAJ46JA4FPSOOQMWP247A3CGPMMTZXWQUA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.242175176 (223h38m36.009858606s) [noise=true , address=81.171.14.70:4028 , piece_id=HEZAKOJDOQEBIFDRGYIXSCUWLPKUBTA6MRKQPR6665ZQXEPVIGQA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.242769247 (223h38m36.010452683s) [noise=true , address=81.171.14.70:4030 , piece_id=QAOOJJY2QL6NZQ3DN4NG5MNQCLPFBEZ73BJ4ESWPHMMHFJZV3I6Q, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.243245042 (223h38m36.010928472s) [noise=true , address=81.171.14.70:4029 , piece_id=XJF5EGWPXCL3PXV6WXEL46JAHMHZQGGR4AU5IOUNQLBG4C3BGQUA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.247956007 (223h38m36.015639437s) [noise=true , address=81.171.14.70:4048 , piece_id=KUYGLZGDWTEMEG637NPTQCKAEOAU42SCT4QKIGTAFTIVTUTMUV6A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.251384119 (223h38m36.019067547s) [noise=true , address=81.171.14.70:4025 , piece_id=EZNZI3X3CQI5B6ZMKVZABPZ5IZHVVS722EXVLWRHEF625CZFGORA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.251410363 (223h38m36.019093791s) [seg_pos={0 0}, upload=232637]: Segment reached optimal threshold of 26 pieces. Nov 20 17:25:11.251626678 (223h38m36.019310112s) [noise=true , address=89.149.201.147:4018 , piece_id=EF6O3MQ6T6Y6SZLNBTHJGH4U52JOUUIT5FJSRBBZB3R3XRH7MDKA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua): protocol: expected piece hash; context canceled; EOF Nov 20 17:25:11.251642292 (223h38m36.019325718s) [noise=true , address=89.149.201.147:4018 , piece_id=EF6O3MQ6T6Y6SZLNBTHJGH4U52JOUUIT5FJSRBBZB3R3XRH7MDKA, seg_pos={0 0}, upload=232637]: Long tail cancellation due to optimal threshold Nov 20 17:25:11.252013645 (223h38m36.019697078s) [noise=true , address=81.171.14.70:4038 , piece_id=GCUUKZQY36F6XWCXJGHP6SSK5OKAH47HLPNBC76GVNNQID4DVLDA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB): protocol: expected piece hash; context canceled; EOF Nov 20 17:25:11.252022652 (223h38m36.019706079s) [noise=true , address=81.171.14.70:4038 , piece_id=GCUUKZQY36F6XWCXJGHP6SSK5OKAH47HLPNBC76GVNNQID4DVLDA, seg_pos={0 0}, upload=232637]: Long tail cancellation due to optimal threshold Nov 20 17:25:11.252045137 (223h38m36.019728563s) [seg_pos={0 0}, upload=232637]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 20 17:25:11.252321599 (223h38m36.020005025s) [upload=232637]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 20 17:25:11.252330210 (223h38m36.020013635s) [upload=232637]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 17:25:11.439953049 (223h38m36.207636477s) [upload=232637]: Done uploading segments... Nov 20 17:25:11.439975748 (223h38m36.207659185s) [upload=232637]: Upload finished. err: <nil> Duration: 2.393785663s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 1.999448061s trace: https://jaeger.dev.storj.io/trace/2978a773c720f2e4 FAIL (2025-11-20 17:30:10 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== load=232637]: Uploading piece... Nov 20 17:25:10.582877896 (223h38m35.350561328s) [noise=true , address=81.171.14.70:4029 , piece_id=XJF5EGWPXCL3PXV6WXEL46JAHMHZQGGR4AU5IOUNQLBG4C3BGQUA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.584580409 (223h38m35.352263843s) [noise=true , address=185.167.204.205:4026 , piece_id=YVFA5OFDVPAN3Q7GP7NIO6MC2N47G2MEWLJ7RVJEIP7UBHBYD5NA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.586378791 (223h38m35.354062219s) [noise=true , address=185.167.204.205:4036 , piece_id=LZSPRFBBIJPCW3IWHNW3SNKX4YTW7RCXTOOLE5SBGR2Z4WJHVCZA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.588641711 (223h38m35.356325142s) [noise=true , address=185.167.204.205:4015 , piece_id=ZC3TZGH2N7NHRNMFGG2SVOS5UGTD4CY4OCLCQL2EEIN6RHEMQKIQ, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.591403897 (223h38m35.359087325s) [noise=true , address=185.167.204.205:4034 , piece_id=MPKSGXYASP3YGAXWHL4PLGA7PJX56ELPS6NPJQIKLQN3GOYAUFDA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.593348617 (223h38m35.361032051s) [noise=true , address=81.171.14.70:4031 , piece_id=SHBKH5K4C7T7D7XMAAD6FDA36VCEJXQILDGEVNJZLTMD6EXLBL5A, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.595567737 (223h38m35.363251162s) [noise=true , address=185.167.204.205:4002 , piece_id=27KMJOJSSJTIHEO5ZKY66GIG66P5QAESU4HHHLTI6KV2WZCFRWYQ, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.597504768 (223h38m35.365188202s) [noise=true , address=81.171.14.70:4028 , piece_id=HEZAKOJDOQEBIFDRGYIXSCUWLPKUBTA6MRKQPR6665ZQXEPVIGQA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.599544427 (223h38m35.367227861s) [noise=true , address=81.171.14.70:4038 , piece_id=GCUUKZQY36F6XWCXJGHP6SSK5OKAH47HLPNBC76GVNNQID4DVLDA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.600434395 (223h38m35.36811783s) [noise=true , address=185.167.204.206:4065 , piece_id=UHVHSM2N5H73YGCYEJ26JWO6IZZ76JTTYWM5ZZZZ2HMERRK7VDWA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.992513100 (223h38m35.760196525s) [upload=232637]: Next returned nil segment Nov 20 17:25:10.992541848 (223h38m35.760225277s) [upload=232637]: Waiting for error group managing segments... Nov 20 17:25:11.164544942 (223h38m35.932228366s) [noise=true , address=89.149.201.147:4022 , piece_id=4NQ4J3UJAABVO3SFMRORPXBTUGYXZMJNUWWTAKFIH3ZD5EVF3UXQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.172767990 (223h38m35.940451418s) [noise=true , address=81.171.14.70:4031 , piece_id=SHBKH5K4C7T7D7XMAAD6FDA36VCEJXQILDGEVNJZLTMD6EXLBL5A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.182086404 (223h38m35.949769833s) [noise=true , address=185.167.204.205:4015 , piece_id=ZC3TZGH2N7NHRNMFGG2SVOS5UGTD4CY4OCLCQL2EEIN6RHEMQKIQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.182810476 (223h38m35.950493902s) [noise=true , address=81.171.14.70:4043 , piece_id=6474UQDNSX4UVDXJWZKSDBB5T47HGTYJZMJNTFK3PEQBR2KAY7RA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.183742078 (223h38m35.951425504s) [noise=true , address=185.167.204.206:4053 , piece_id=AIF5XPKJPTGUA7ENKUUTPPOKQCECEKQPPGEMAUOT5QAISCVMO22Q, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.184529255 (223h38m35.952212684s) [noise=true , address=81.171.14.70:4044 , piece_id=RICD4ZMMWVGLS6UTFKKZFUNKE2LI35QQ6VF4FYRMLDUMQDOQCCXA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.185123500 (223h38m35.952806926s) [noise=true , address=185.167.204.206:4066 , piece_id=H52XDJBLWKNNCE25VSBRNUH6BMW3JPBOFGQSEKF4CI3HBKCQSJZQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.186964052 (223h38m35.954647478s) [noise=true , address=185.167.204.205:4028 , piece_id=FVBJ6WVHKSN43OVNMH66KPBREGS2EWGD2EWCVJQX2WVMI27MPC7A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.191410051 (223h38m35.959093477s) [noise=true , address=185.167.204.206:4054 , piece_id=LCBZUECG7VIQOE3X4VT6AT4HEQEEOTHEFBPSRXG7NUJC4PCGQ72Q, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.194616300 (223h38m35.962299725s) [noise=true , address=89.149.201.147:4001 , piece_id=7YTUOPW4BA4KDNJAXDHNUENBDIXNGZZW6576XQD5D3ISQXKBTYTQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.195124666 (223h38m35.962808091s) [noise=true , address=81.171.14.70:4027 , piece_id=OOS4I4CDV2TN42MWMHZBRCQWXGYXLHJTTUONDMTAHSQJKKPROYJA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.196224298 (223h38m35.963907724s) [noise=true , address=89.149.201.147:4005 , piece_id=U64XSFTL2TDVWNNOUQ6ULUGBPNU5XGQ6SI3HU7ZP6PPLZ7PZVRQA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.198703124 (223h38m35.966386601s) [noise=true , address=185.167.204.206:4047 , piece_id=YZFK5OW7TWWIJ6FLA2XMAU3YURPKT2U5FGZYH6YLRNHUUDIV5N7A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.201965022 (223h38m35.969648447s) [noise=true , address=185.167.204.205:4034 , piece_id=MPKSGXYASP3YGAXWHL4PLGA7PJX56ELPS6NPJQIKLQN3GOYAUFDA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.202483607 (223h38m35.970167033s) [noise=true , address=185.167.204.205:4002 , piece_id=27KMJOJSSJTIHEO5ZKY66GIG66P5QAESU4HHHLTI6KV2WZCFRWYQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.211202242 (223h38m35.978885672s) [noise=true , address=185.167.204.205:4036 , piece_id=LZSPRFBBIJPCW3IWHNW3SNKX4YTW7RCXTOOLE5SBGR2Z4WJHVCZA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.216399102 (223h38m35.984082529s) [noise=true , address=185.167.204.206:4046 , piece_id=VY3VEVR5VIHEH3MWJKDYS2UM62EP7ELZU3JI346NDESEORJIPZDQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.226445839 (223h38m35.994129265s) [noise=true , address=185.167.204.205:4026 , piece_id=YVFA5OFDVPAN3Q7GP7NIO6MC2N47G2MEWLJ7RVJEIP7UBHBYD5NA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.240411560 (223h38m36.008094989s) [noise=true , address=185.167.204.206:4065 , piece_id=UHVHSM2N5H73YGCYEJ26JWO6IZZ76JTTYWM5ZZZZ2HMERRK7VDWA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.240958840 (223h38m36.008642266s) [noise=true , address=81.171.14.70:4047 , piece_id=D46YVEDXW2RT6OUYO73PHKREJF55SOCRWX6AMJ6RUWUC7G67426A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.241565965 (223h38m36.009249392s) [noise=true , address=81.171.14.70:4036 , piece_id=RNDJKD3UBSBBCUSY6AMAJ46JA4FPSOOQMWP247A3CGPMMTZXWQUA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.242175176 (223h38m36.009858606s) [noise=true , address=81.171.14.70:4028 , piece_id=HEZAKOJDOQEBIFDRGYIXSCUWLPKUBTA6MRKQPR6665ZQXEPVIGQA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.242769247 (223h38m36.010452683s) [noise=true , address=81.171.14.70:4030 , piece_id=QAOOJJY2QL6NZQ3DN4NG5MNQCLPFBEZ73BJ4ESWPHMMHFJZV3I6Q, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.243245042 (223h38m36.010928472s) [noise=true , address=81.171.14.70:4029 , piece_id=XJF5EGWPXCL3PXV6WXEL46JAHMHZQGGR4AU5IOUNQLBG4C3BGQUA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.247956007 (223h38m36.015639437s) [noise=true , address=81.171.14.70:4048 , piece_id=KUYGLZGDWTEMEG637NPTQCKAEOAU42SCT4QKIGTAFTIVTUTMUV6A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.251384119 (223h38m36.019067547s) [noise=true , address=81.171.14.70:4025 , piece_id=EZNZI3X3CQI5B6ZMKVZABPZ5IZHVVS722EXVLWRHEF625CZFGORA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.251410363 (223h38m36.019093791s) [seg_pos={0 0}, upload=232637]: Segment reached optimal threshold of 26 pieces. Nov 20 17:25:11.251626678 (223h38m36.019310112s) [noise=true , address=89.149.201.147:4018 , piece_id=EF6O3MQ6T6Y6SZLNBTHJGH4U52JOUUIT5FJSRBBZB3R3XRH7MDKA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua): protocol: expected piece hash; context canceled; EOF Nov 20 17:25:11.251642292 (223h38m36.019325718s) [noise=true , address=89.149.201.147:4018 , piece_id=EF6O3MQ6T6Y6SZLNBTHJGH4U52JOUUIT5FJSRBBZB3R3XRH7MDKA, seg_pos={0 0}, upload=232637]: Long tail cancellation due to optimal threshold Nov 20 17:25:11.252013645 (223h38m36.019697078s) [noise=true , address=81.171.14.70:4038 , piece_id=GCUUKZQY36F6XWCXJGHP6SSK5OKAH47HLPNBC76GVNNQID4DVLDA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB): protocol: expected piece hash; context canceled; EOF Nov 20 17:25:11.252022652 (223h38m36.019706079s) [noise=true , address=81.171.14.70:4038 , piece_id=GCUUKZQY36F6XWCXJGHP6SSK5OKAH47HLPNBC76GVNNQID4DVLDA, seg_pos={0 0}, upload=232637]: Long tail cancellation due to optimal threshold Nov 20 17:25:11.252045137 (223h38m36.019728563s) [seg_pos={0 0}, upload=232637]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 20 17:25:11.252321599 (223h38m36.020005025s) [upload=232637]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 20 17:25:11.252330210 (223h38m36.020013635s) [upload=232637]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 17:25:11.439953049 (223h38m36.207636477s) [upload=232637]: Done uploading segments... Nov 20 17:25:11.439975748 (223h38m36.207659185s) [upload=232637]: Upload finished. err: <nil> Duration: 1.999476511s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 250.375956ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:29:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 250.399779ms core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.497114943s trace: https://jaeger.dev.storj.io/trace/2978a773c720f2e4 FAIL (2025-11-20 17:29:44 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== load=232637]: Uploading piece... Nov 20 17:25:10.582877896 (223h38m35.350561328s) [noise=true , address=81.171.14.70:4029 , piece_id=XJF5EGWPXCL3PXV6WXEL46JAHMHZQGGR4AU5IOUNQLBG4C3BGQUA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.584580409 (223h38m35.352263843s) [noise=true , address=185.167.204.205:4026 , piece_id=YVFA5OFDVPAN3Q7GP7NIO6MC2N47G2MEWLJ7RVJEIP7UBHBYD5NA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.586378791 (223h38m35.354062219s) [noise=true , address=185.167.204.205:4036 , piece_id=LZSPRFBBIJPCW3IWHNW3SNKX4YTW7RCXTOOLE5SBGR2Z4WJHVCZA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.588641711 (223h38m35.356325142s) [noise=true , address=185.167.204.205:4015 , piece_id=ZC3TZGH2N7NHRNMFGG2SVOS5UGTD4CY4OCLCQL2EEIN6RHEMQKIQ, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.591403897 (223h38m35.359087325s) [noise=true , address=185.167.204.205:4034 , piece_id=MPKSGXYASP3YGAXWHL4PLGA7PJX56ELPS6NPJQIKLQN3GOYAUFDA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.593348617 (223h38m35.361032051s) [noise=true , address=81.171.14.70:4031 , piece_id=SHBKH5K4C7T7D7XMAAD6FDA36VCEJXQILDGEVNJZLTMD6EXLBL5A, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.595567737 (223h38m35.363251162s) [noise=true , address=185.167.204.205:4002 , piece_id=27KMJOJSSJTIHEO5ZKY66GIG66P5QAESU4HHHLTI6KV2WZCFRWYQ, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.597504768 (223h38m35.365188202s) [noise=true , address=81.171.14.70:4028 , piece_id=HEZAKOJDOQEBIFDRGYIXSCUWLPKUBTA6MRKQPR6665ZQXEPVIGQA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.599544427 (223h38m35.367227861s) [noise=true , address=81.171.14.70:4038 , piece_id=GCUUKZQY36F6XWCXJGHP6SSK5OKAH47HLPNBC76GVNNQID4DVLDA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.600434395 (223h38m35.36811783s) [noise=true , address=185.167.204.206:4065 , piece_id=UHVHSM2N5H73YGCYEJ26JWO6IZZ76JTTYWM5ZZZZ2HMERRK7VDWA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.992513100 (223h38m35.760196525s) [upload=232637]: Next returned nil segment Nov 20 17:25:10.992541848 (223h38m35.760225277s) [upload=232637]: Waiting for error group managing segments... Nov 20 17:25:11.164544942 (223h38m35.932228366s) [noise=true , address=89.149.201.147:4022 , piece_id=4NQ4J3UJAABVO3SFMRORPXBTUGYXZMJNUWWTAKFIH3ZD5EVF3UXQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.172767990 (223h38m35.940451418s) [noise=true , address=81.171.14.70:4031 , piece_id=SHBKH5K4C7T7D7XMAAD6FDA36VCEJXQILDGEVNJZLTMD6EXLBL5A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.182086404 (223h38m35.949769833s) [noise=true , address=185.167.204.205:4015 , piece_id=ZC3TZGH2N7NHRNMFGG2SVOS5UGTD4CY4OCLCQL2EEIN6RHEMQKIQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.182810476 (223h38m35.950493902s) [noise=true , address=81.171.14.70:4043 , piece_id=6474UQDNSX4UVDXJWZKSDBB5T47HGTYJZMJNTFK3PEQBR2KAY7RA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.183742078 (223h38m35.951425504s) [noise=true , address=185.167.204.206:4053 , piece_id=AIF5XPKJPTGUA7ENKUUTPPOKQCECEKQPPGEMAUOT5QAISCVMO22Q, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.184529255 (223h38m35.952212684s) [noise=true , address=81.171.14.70:4044 , piece_id=RICD4ZMMWVGLS6UTFKKZFUNKE2LI35QQ6VF4FYRMLDUMQDOQCCXA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.185123500 (223h38m35.952806926s) [noise=true , address=185.167.204.206:4066 , piece_id=H52XDJBLWKNNCE25VSBRNUH6BMW3JPBOFGQSEKF4CI3HBKCQSJZQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.186964052 (223h38m35.954647478s) [noise=true , address=185.167.204.205:4028 , piece_id=FVBJ6WVHKSN43OVNMH66KPBREGS2EWGD2EWCVJQX2WVMI27MPC7A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.191410051 (223h38m35.959093477s) [noise=true , address=185.167.204.206:4054 , piece_id=LCBZUECG7VIQOE3X4VT6AT4HEQEEOTHEFBPSRXG7NUJC4PCGQ72Q, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.194616300 (223h38m35.962299725s) [noise=true , address=89.149.201.147:4001 , piece_id=7YTUOPW4BA4KDNJAXDHNUENBDIXNGZZW6576XQD5D3ISQXKBTYTQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.195124666 (223h38m35.962808091s) [noise=true , address=81.171.14.70:4027 , piece_id=OOS4I4CDV2TN42MWMHZBRCQWXGYXLHJTTUONDMTAHSQJKKPROYJA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.196224298 (223h38m35.963907724s) [noise=true , address=89.149.201.147:4005 , piece_id=U64XSFTL2TDVWNNOUQ6ULUGBPNU5XGQ6SI3HU7ZP6PPLZ7PZVRQA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.198703124 (223h38m35.966386601s) [noise=true , address=185.167.204.206:4047 , piece_id=YZFK5OW7TWWIJ6FLA2XMAU3YURPKT2U5FGZYH6YLRNHUUDIV5N7A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.201965022 (223h38m35.969648447s) [noise=true , address=185.167.204.205:4034 , piece_id=MPKSGXYASP3YGAXWHL4PLGA7PJX56ELPS6NPJQIKLQN3GOYAUFDA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.202483607 (223h38m35.970167033s) [noise=true , address=185.167.204.205:4002 , piece_id=27KMJOJSSJTIHEO5ZKY66GIG66P5QAESU4HHHLTI6KV2WZCFRWYQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.211202242 (223h38m35.978885672s) [noise=true , address=185.167.204.205:4036 , piece_id=LZSPRFBBIJPCW3IWHNW3SNKX4YTW7RCXTOOLE5SBGR2Z4WJHVCZA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.216399102 (223h38m35.984082529s) [noise=true , address=185.167.204.206:4046 , piece_id=VY3VEVR5VIHEH3MWJKDYS2UM62EP7ELZU3JI346NDESEORJIPZDQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.226445839 (223h38m35.994129265s) [noise=true , address=185.167.204.205:4026 , piece_id=YVFA5OFDVPAN3Q7GP7NIO6MC2N47G2MEWLJ7RVJEIP7UBHBYD5NA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.240411560 (223h38m36.008094989s) [noise=true , address=185.167.204.206:4065 , piece_id=UHVHSM2N5H73YGCYEJ26JWO6IZZ76JTTYWM5ZZZZ2HMERRK7VDWA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.240958840 (223h38m36.008642266s) [noise=true , address=81.171.14.70:4047 , piece_id=D46YVEDXW2RT6OUYO73PHKREJF55SOCRWX6AMJ6RUWUC7G67426A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.241565965 (223h38m36.009249392s) [noise=true , address=81.171.14.70:4036 , piece_id=RNDJKD3UBSBBCUSY6AMAJ46JA4FPSOOQMWP247A3CGPMMTZXWQUA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.242175176 (223h38m36.009858606s) [noise=true , address=81.171.14.70:4028 , piece_id=HEZAKOJDOQEBIFDRGYIXSCUWLPKUBTA6MRKQPR6665ZQXEPVIGQA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.242769247 (223h38m36.010452683s) [noise=true , address=81.171.14.70:4030 , piece_id=QAOOJJY2QL6NZQ3DN4NG5MNQCLPFBEZ73BJ4ESWPHMMHFJZV3I6Q, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.243245042 (223h38m36.010928472s) [noise=true , address=81.171.14.70:4029 , piece_id=XJF5EGWPXCL3PXV6WXEL46JAHMHZQGGR4AU5IOUNQLBG4C3BGQUA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.247956007 (223h38m36.015639437s) [noise=true , address=81.171.14.70:4048 , piece_id=KUYGLZGDWTEMEG637NPTQCKAEOAU42SCT4QKIGTAFTIVTUTMUV6A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.251384119 (223h38m36.019067547s) [noise=true , address=81.171.14.70:4025 , piece_id=EZNZI3X3CQI5B6ZMKVZABPZ5IZHVVS722EXVLWRHEF625CZFGORA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.251410363 (223h38m36.019093791s) [seg_pos={0 0}, upload=232637]: Segment reached optimal threshold of 26 pieces. Nov 20 17:25:11.251626678 (223h38m36.019310112s) [noise=true , address=89.149.201.147:4018 , piece_id=EF6O3MQ6T6Y6SZLNBTHJGH4U52JOUUIT5FJSRBBZB3R3XRH7MDKA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua): protocol: expected piece hash; context canceled; EOF Nov 20 17:25:11.251642292 (223h38m36.019325718s) [noise=true , address=89.149.201.147:4018 , piece_id=EF6O3MQ6T6Y6SZLNBTHJGH4U52JOUUIT5FJSRBBZB3R3XRH7MDKA, seg_pos={0 0}, upload=232637]: Long tail cancellation due to optimal threshold Nov 20 17:25:11.252013645 (223h38m36.019697078s) [noise=true , address=81.171.14.70:4038 , piece_id=GCUUKZQY36F6XWCXJGHP6SSK5OKAH47HLPNBC76GVNNQID4DVLDA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB): protocol: expected piece hash; context canceled; EOF Nov 20 17:25:11.252022652 (223h38m36.019706079s) [noise=true , address=81.171.14.70:4038 , piece_id=GCUUKZQY36F6XWCXJGHP6SSK5OKAH47HLPNBC76GVNNQID4DVLDA, seg_pos={0 0}, upload=232637]: Long tail cancellation due to optimal threshold Nov 20 17:25:11.252045137 (223h38m36.019728563s) [seg_pos={0 0}, upload=232637]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 20 17:25:11.252321599 (223h38m36.020005025s) [upload=232637]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 20 17:25:11.252330210 (223h38m36.020013635s) [upload=232637]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 17:25:11.439953049 (223h38m36.207636477s) [upload=232637]: Done uploading segments... Nov 20 17:25:11.439975748 (223h38m36.207659185s) [upload=232637]: Upload finished. err: <nil> Duration: 2.497143675s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 368.406766ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:28:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 368.433338ms core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.454756423s trace: https://jaeger.dev.storj.io/trace/2978a773c720f2e4 FAIL (2025-11-20 17:28:46 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== load=232637]: Uploading piece... Nov 20 17:25:10.582877896 (223h38m35.350561328s) [noise=true , address=81.171.14.70:4029 , piece_id=XJF5EGWPXCL3PXV6WXEL46JAHMHZQGGR4AU5IOUNQLBG4C3BGQUA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.584580409 (223h38m35.352263843s) [noise=true , address=185.167.204.205:4026 , piece_id=YVFA5OFDVPAN3Q7GP7NIO6MC2N47G2MEWLJ7RVJEIP7UBHBYD5NA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.586378791 (223h38m35.354062219s) [noise=true , address=185.167.204.205:4036 , piece_id=LZSPRFBBIJPCW3IWHNW3SNKX4YTW7RCXTOOLE5SBGR2Z4WJHVCZA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.588641711 (223h38m35.356325142s) [noise=true , address=185.167.204.205:4015 , piece_id=ZC3TZGH2N7NHRNMFGG2SVOS5UGTD4CY4OCLCQL2EEIN6RHEMQKIQ, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.591403897 (223h38m35.359087325s) [noise=true , address=185.167.204.205:4034 , piece_id=MPKSGXYASP3YGAXWHL4PLGA7PJX56ELPS6NPJQIKLQN3GOYAUFDA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.593348617 (223h38m35.361032051s) [noise=true , address=81.171.14.70:4031 , piece_id=SHBKH5K4C7T7D7XMAAD6FDA36VCEJXQILDGEVNJZLTMD6EXLBL5A, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.595567737 (223h38m35.363251162s) [noise=true , address=185.167.204.205:4002 , piece_id=27KMJOJSSJTIHEO5ZKY66GIG66P5QAESU4HHHLTI6KV2WZCFRWYQ, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.597504768 (223h38m35.365188202s) [noise=true , address=81.171.14.70:4028 , piece_id=HEZAKOJDOQEBIFDRGYIXSCUWLPKUBTA6MRKQPR6665ZQXEPVIGQA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.599544427 (223h38m35.367227861s) [noise=true , address=81.171.14.70:4038 , piece_id=GCUUKZQY36F6XWCXJGHP6SSK5OKAH47HLPNBC76GVNNQID4DVLDA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.600434395 (223h38m35.36811783s) [noise=true , address=185.167.204.206:4065 , piece_id=UHVHSM2N5H73YGCYEJ26JWO6IZZ76JTTYWM5ZZZZ2HMERRK7VDWA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.992513100 (223h38m35.760196525s) [upload=232637]: Next returned nil segment Nov 20 17:25:10.992541848 (223h38m35.760225277s) [upload=232637]: Waiting for error group managing segments... Nov 20 17:25:11.164544942 (223h38m35.932228366s) [noise=true , address=89.149.201.147:4022 , piece_id=4NQ4J3UJAABVO3SFMRORPXBTUGYXZMJNUWWTAKFIH3ZD5EVF3UXQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.172767990 (223h38m35.940451418s) [noise=true , address=81.171.14.70:4031 , piece_id=SHBKH5K4C7T7D7XMAAD6FDA36VCEJXQILDGEVNJZLTMD6EXLBL5A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.182086404 (223h38m35.949769833s) [noise=true , address=185.167.204.205:4015 , piece_id=ZC3TZGH2N7NHRNMFGG2SVOS5UGTD4CY4OCLCQL2EEIN6RHEMQKIQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.182810476 (223h38m35.950493902s) [noise=true , address=81.171.14.70:4043 , piece_id=6474UQDNSX4UVDXJWZKSDBB5T47HGTYJZMJNTFK3PEQBR2KAY7RA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.183742078 (223h38m35.951425504s) [noise=true , address=185.167.204.206:4053 , piece_id=AIF5XPKJPTGUA7ENKUUTPPOKQCECEKQPPGEMAUOT5QAISCVMO22Q, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.184529255 (223h38m35.952212684s) [noise=true , address=81.171.14.70:4044 , piece_id=RICD4ZMMWVGLS6UTFKKZFUNKE2LI35QQ6VF4FYRMLDUMQDOQCCXA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.185123500 (223h38m35.952806926s) [noise=true , address=185.167.204.206:4066 , piece_id=H52XDJBLWKNNCE25VSBRNUH6BMW3JPBOFGQSEKF4CI3HBKCQSJZQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.186964052 (223h38m35.954647478s) [noise=true , address=185.167.204.205:4028 , piece_id=FVBJ6WVHKSN43OVNMH66KPBREGS2EWGD2EWCVJQX2WVMI27MPC7A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.191410051 (223h38m35.959093477s) [noise=true , address=185.167.204.206:4054 , piece_id=LCBZUECG7VIQOE3X4VT6AT4HEQEEOTHEFBPSRXG7NUJC4PCGQ72Q, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.194616300 (223h38m35.962299725s) [noise=true , address=89.149.201.147:4001 , piece_id=7YTUOPW4BA4KDNJAXDHNUENBDIXNGZZW6576XQD5D3ISQXKBTYTQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.195124666 (223h38m35.962808091s) [noise=true , address=81.171.14.70:4027 , piece_id=OOS4I4CDV2TN42MWMHZBRCQWXGYXLHJTTUONDMTAHSQJKKPROYJA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.196224298 (223h38m35.963907724s) [noise=true , address=89.149.201.147:4005 , piece_id=U64XSFTL2TDVWNNOUQ6ULUGBPNU5XGQ6SI3HU7ZP6PPLZ7PZVRQA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.198703124 (223h38m35.966386601s) [noise=true , address=185.167.204.206:4047 , piece_id=YZFK5OW7TWWIJ6FLA2XMAU3YURPKT2U5FGZYH6YLRNHUUDIV5N7A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.201965022 (223h38m35.969648447s) [noise=true , address=185.167.204.205:4034 , piece_id=MPKSGXYASP3YGAXWHL4PLGA7PJX56ELPS6NPJQIKLQN3GOYAUFDA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.202483607 (223h38m35.970167033s) [noise=true , address=185.167.204.205:4002 , piece_id=27KMJOJSSJTIHEO5ZKY66GIG66P5QAESU4HHHLTI6KV2WZCFRWYQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.211202242 (223h38m35.978885672s) [noise=true , address=185.167.204.205:4036 , piece_id=LZSPRFBBIJPCW3IWHNW3SNKX4YTW7RCXTOOLE5SBGR2Z4WJHVCZA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.216399102 (223h38m35.984082529s) [noise=true , address=185.167.204.206:4046 , piece_id=VY3VEVR5VIHEH3MWJKDYS2UM62EP7ELZU3JI346NDESEORJIPZDQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.226445839 (223h38m35.994129265s) [noise=true , address=185.167.204.205:4026 , piece_id=YVFA5OFDVPAN3Q7GP7NIO6MC2N47G2MEWLJ7RVJEIP7UBHBYD5NA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.240411560 (223h38m36.008094989s) [noise=true , address=185.167.204.206:4065 , piece_id=UHVHSM2N5H73YGCYEJ26JWO6IZZ76JTTYWM5ZZZZ2HMERRK7VDWA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.240958840 (223h38m36.008642266s) [noise=true , address=81.171.14.70:4047 , piece_id=D46YVEDXW2RT6OUYO73PHKREJF55SOCRWX6AMJ6RUWUC7G67426A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.241565965 (223h38m36.009249392s) [noise=true , address=81.171.14.70:4036 , piece_id=RNDJKD3UBSBBCUSY6AMAJ46JA4FPSOOQMWP247A3CGPMMTZXWQUA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.242175176 (223h38m36.009858606s) [noise=true , address=81.171.14.70:4028 , piece_id=HEZAKOJDOQEBIFDRGYIXSCUWLPKUBTA6MRKQPR6665ZQXEPVIGQA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.242769247 (223h38m36.010452683s) [noise=true , address=81.171.14.70:4030 , piece_id=QAOOJJY2QL6NZQ3DN4NG5MNQCLPFBEZ73BJ4ESWPHMMHFJZV3I6Q, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.243245042 (223h38m36.010928472s) [noise=true , address=81.171.14.70:4029 , piece_id=XJF5EGWPXCL3PXV6WXEL46JAHMHZQGGR4AU5IOUNQLBG4C3BGQUA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.247956007 (223h38m36.015639437s) [noise=true , address=81.171.14.70:4048 , piece_id=KUYGLZGDWTEMEG637NPTQCKAEOAU42SCT4QKIGTAFTIVTUTMUV6A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.251384119 (223h38m36.019067547s) [noise=true , address=81.171.14.70:4025 , piece_id=EZNZI3X3CQI5B6ZMKVZABPZ5IZHVVS722EXVLWRHEF625CZFGORA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.251410363 (223h38m36.019093791s) [seg_pos={0 0}, upload=232637]: Segment reached optimal threshold of 26 pieces. Nov 20 17:25:11.251626678 (223h38m36.019310112s) [noise=true , address=89.149.201.147:4018 , piece_id=EF6O3MQ6T6Y6SZLNBTHJGH4U52JOUUIT5FJSRBBZB3R3XRH7MDKA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua): protocol: expected piece hash; context canceled; EOF Nov 20 17:25:11.251642292 (223h38m36.019325718s) [noise=true , address=89.149.201.147:4018 , piece_id=EF6O3MQ6T6Y6SZLNBTHJGH4U52JOUUIT5FJSRBBZB3R3XRH7MDKA, seg_pos={0 0}, upload=232637]: Long tail cancellation due to optimal threshold Nov 20 17:25:11.252013645 (223h38m36.019697078s) [noise=true , address=81.171.14.70:4038 , piece_id=GCUUKZQY36F6XWCXJGHP6SSK5OKAH47HLPNBC76GVNNQID4DVLDA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB): protocol: expected piece hash; context canceled; EOF Nov 20 17:25:11.252022652 (223h38m36.019706079s) [noise=true , address=81.171.14.70:4038 , piece_id=GCUUKZQY36F6XWCXJGHP6SSK5OKAH47HLPNBC76GVNNQID4DVLDA, seg_pos={0 0}, upload=232637]: Long tail cancellation due to optimal threshold Nov 20 17:25:11.252045137 (223h38m36.019728563s) [seg_pos={0 0}, upload=232637]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 20 17:25:11.252321599 (223h38m36.020005025s) [upload=232637]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 20 17:25:11.252330210 (223h38m36.020013635s) [upload=232637]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 17:25:11.439953049 (223h38m36.207636477s) [upload=232637]: Done uploading segments... Nov 20 17:25:11.439975748 (223h38m36.207659185s) [upload=232637]: Upload finished. err: <nil> Duration: 2.454854495s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 343.427038ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:28:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 343.517709ms core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 1.996895769s trace: https://jaeger.dev.storj.io/trace/2978a773c720f2e4 FAIL (2025-11-20 17:28:10 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== load=232637]: Uploading piece... Nov 20 17:25:10.582877896 (223h38m35.350561328s) [noise=true , address=81.171.14.70:4029 , piece_id=XJF5EGWPXCL3PXV6WXEL46JAHMHZQGGR4AU5IOUNQLBG4C3BGQUA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.584580409 (223h38m35.352263843s) [noise=true , address=185.167.204.205:4026 , piece_id=YVFA5OFDVPAN3Q7GP7NIO6MC2N47G2MEWLJ7RVJEIP7UBHBYD5NA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.586378791 (223h38m35.354062219s) [noise=true , address=185.167.204.205:4036 , piece_id=LZSPRFBBIJPCW3IWHNW3SNKX4YTW7RCXTOOLE5SBGR2Z4WJHVCZA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.588641711 (223h38m35.356325142s) [noise=true , address=185.167.204.205:4015 , piece_id=ZC3TZGH2N7NHRNMFGG2SVOS5UGTD4CY4OCLCQL2EEIN6RHEMQKIQ, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.591403897 (223h38m35.359087325s) [noise=true , address=185.167.204.205:4034 , piece_id=MPKSGXYASP3YGAXWHL4PLGA7PJX56ELPS6NPJQIKLQN3GOYAUFDA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.593348617 (223h38m35.361032051s) [noise=true , address=81.171.14.70:4031 , piece_id=SHBKH5K4C7T7D7XMAAD6FDA36VCEJXQILDGEVNJZLTMD6EXLBL5A, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.595567737 (223h38m35.363251162s) [noise=true , address=185.167.204.205:4002 , piece_id=27KMJOJSSJTIHEO5ZKY66GIG66P5QAESU4HHHLTI6KV2WZCFRWYQ, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.597504768 (223h38m35.365188202s) [noise=true , address=81.171.14.70:4028 , piece_id=HEZAKOJDOQEBIFDRGYIXSCUWLPKUBTA6MRKQPR6665ZQXEPVIGQA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.599544427 (223h38m35.367227861s) [noise=true , address=81.171.14.70:4038 , piece_id=GCUUKZQY36F6XWCXJGHP6SSK5OKAH47HLPNBC76GVNNQID4DVLDA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.600434395 (223h38m35.36811783s) [noise=true , address=185.167.204.206:4065 , piece_id=UHVHSM2N5H73YGCYEJ26JWO6IZZ76JTTYWM5ZZZZ2HMERRK7VDWA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.992513100 (223h38m35.760196525s) [upload=232637]: Next returned nil segment Nov 20 17:25:10.992541848 (223h38m35.760225277s) [upload=232637]: Waiting for error group managing segments... Nov 20 17:25:11.164544942 (223h38m35.932228366s) [noise=true , address=89.149.201.147:4022 , piece_id=4NQ4J3UJAABVO3SFMRORPXBTUGYXZMJNUWWTAKFIH3ZD5EVF3UXQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.172767990 (223h38m35.940451418s) [noise=true , address=81.171.14.70:4031 , piece_id=SHBKH5K4C7T7D7XMAAD6FDA36VCEJXQILDGEVNJZLTMD6EXLBL5A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.182086404 (223h38m35.949769833s) [noise=true , address=185.167.204.205:4015 , piece_id=ZC3TZGH2N7NHRNMFGG2SVOS5UGTD4CY4OCLCQL2EEIN6RHEMQKIQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.182810476 (223h38m35.950493902s) [noise=true , address=81.171.14.70:4043 , piece_id=6474UQDNSX4UVDXJWZKSDBB5T47HGTYJZMJNTFK3PEQBR2KAY7RA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.183742078 (223h38m35.951425504s) [noise=true , address=185.167.204.206:4053 , piece_id=AIF5XPKJPTGUA7ENKUUTPPOKQCECEKQPPGEMAUOT5QAISCVMO22Q, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.184529255 (223h38m35.952212684s) [noise=true , address=81.171.14.70:4044 , piece_id=RICD4ZMMWVGLS6UTFKKZFUNKE2LI35QQ6VF4FYRMLDUMQDOQCCXA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.185123500 (223h38m35.952806926s) [noise=true , address=185.167.204.206:4066 , piece_id=H52XDJBLWKNNCE25VSBRNUH6BMW3JPBOFGQSEKF4CI3HBKCQSJZQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.186964052 (223h38m35.954647478s) [noise=true , address=185.167.204.205:4028 , piece_id=FVBJ6WVHKSN43OVNMH66KPBREGS2EWGD2EWCVJQX2WVMI27MPC7A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.191410051 (223h38m35.959093477s) [noise=true , address=185.167.204.206:4054 , piece_id=LCBZUECG7VIQOE3X4VT6AT4HEQEEOTHEFBPSRXG7NUJC4PCGQ72Q, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.194616300 (223h38m35.962299725s) [noise=true , address=89.149.201.147:4001 , piece_id=7YTUOPW4BA4KDNJAXDHNUENBDIXNGZZW6576XQD5D3ISQXKBTYTQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.195124666 (223h38m35.962808091s) [noise=true , address=81.171.14.70:4027 , piece_id=OOS4I4CDV2TN42MWMHZBRCQWXGYXLHJTTUONDMTAHSQJKKPROYJA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.196224298 (223h38m35.963907724s) [noise=true , address=89.149.201.147:4005 , piece_id=U64XSFTL2TDVWNNOUQ6ULUGBPNU5XGQ6SI3HU7ZP6PPLZ7PZVRQA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.198703124 (223h38m35.966386601s) [noise=true , address=185.167.204.206:4047 , piece_id=YZFK5OW7TWWIJ6FLA2XMAU3YURPKT2U5FGZYH6YLRNHUUDIV5N7A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.201965022 (223h38m35.969648447s) [noise=true , address=185.167.204.205:4034 , piece_id=MPKSGXYASP3YGAXWHL4PLGA7PJX56ELPS6NPJQIKLQN3GOYAUFDA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.202483607 (223h38m35.970167033s) [noise=true , address=185.167.204.205:4002 , piece_id=27KMJOJSSJTIHEO5ZKY66GIG66P5QAESU4HHHLTI6KV2WZCFRWYQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.211202242 (223h38m35.978885672s) [noise=true , address=185.167.204.205:4036 , piece_id=LZSPRFBBIJPCW3IWHNW3SNKX4YTW7RCXTOOLE5SBGR2Z4WJHVCZA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.216399102 (223h38m35.984082529s) [noise=true , address=185.167.204.206:4046 , piece_id=VY3VEVR5VIHEH3MWJKDYS2UM62EP7ELZU3JI346NDESEORJIPZDQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.226445839 (223h38m35.994129265s) [noise=true , address=185.167.204.205:4026 , piece_id=YVFA5OFDVPAN3Q7GP7NIO6MC2N47G2MEWLJ7RVJEIP7UBHBYD5NA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.240411560 (223h38m36.008094989s) [noise=true , address=185.167.204.206:4065 , piece_id=UHVHSM2N5H73YGCYEJ26JWO6IZZ76JTTYWM5ZZZZ2HMERRK7VDWA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.240958840 (223h38m36.008642266s) [noise=true , address=81.171.14.70:4047 , piece_id=D46YVEDXW2RT6OUYO73PHKREJF55SOCRWX6AMJ6RUWUC7G67426A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.241565965 (223h38m36.009249392s) [noise=true , address=81.171.14.70:4036 , piece_id=RNDJKD3UBSBBCUSY6AMAJ46JA4FPSOOQMWP247A3CGPMMTZXWQUA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.242175176 (223h38m36.009858606s) [noise=true , address=81.171.14.70:4028 , piece_id=HEZAKOJDOQEBIFDRGYIXSCUWLPKUBTA6MRKQPR6665ZQXEPVIGQA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.242769247 (223h38m36.010452683s) [noise=true , address=81.171.14.70:4030 , piece_id=QAOOJJY2QL6NZQ3DN4NG5MNQCLPFBEZ73BJ4ESWPHMMHFJZV3I6Q, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.243245042 (223h38m36.010928472s) [noise=true , address=81.171.14.70:4029 , piece_id=XJF5EGWPXCL3PXV6WXEL46JAHMHZQGGR4AU5IOUNQLBG4C3BGQUA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.247956007 (223h38m36.015639437s) [noise=true , address=81.171.14.70:4048 , piece_id=KUYGLZGDWTEMEG637NPTQCKAEOAU42SCT4QKIGTAFTIVTUTMUV6A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.251384119 (223h38m36.019067547s) [noise=true , address=81.171.14.70:4025 , piece_id=EZNZI3X3CQI5B6ZMKVZABPZ5IZHVVS722EXVLWRHEF625CZFGORA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.251410363 (223h38m36.019093791s) [seg_pos={0 0}, upload=232637]: Segment reached optimal threshold of 26 pieces. Nov 20 17:25:11.251626678 (223h38m36.019310112s) [noise=true , address=89.149.201.147:4018 , piece_id=EF6O3MQ6T6Y6SZLNBTHJGH4U52JOUUIT5FJSRBBZB3R3XRH7MDKA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua): protocol: expected piece hash; context canceled; EOF Nov 20 17:25:11.251642292 (223h38m36.019325718s) [noise=true , address=89.149.201.147:4018 , piece_id=EF6O3MQ6T6Y6SZLNBTHJGH4U52JOUUIT5FJSRBBZB3R3XRH7MDKA, seg_pos={0 0}, upload=232637]: Long tail cancellation due to optimal threshold Nov 20 17:25:11.252013645 (223h38m36.019697078s) [noise=true , address=81.171.14.70:4038 , piece_id=GCUUKZQY36F6XWCXJGHP6SSK5OKAH47HLPNBC76GVNNQID4DVLDA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB): protocol: expected piece hash; context canceled; EOF Nov 20 17:25:11.252022652 (223h38m36.019706079s) [noise=true , address=81.171.14.70:4038 , piece_id=GCUUKZQY36F6XWCXJGHP6SSK5OKAH47HLPNBC76GVNNQID4DVLDA, seg_pos={0 0}, upload=232637]: Long tail cancellation due to optimal threshold Nov 20 17:25:11.252045137 (223h38m36.019728563s) [seg_pos={0 0}, upload=232637]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 20 17:25:11.252321599 (223h38m36.020005025s) [upload=232637]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 20 17:25:11.252330210 (223h38m36.020013635s) [upload=232637]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 17:25:11.439953049 (223h38m36.207636477s) [upload=232637]: Done uploading segments... Nov 20 17:25:11.439975748 (223h38m36.207659185s) [upload=232637]: Upload finished. err: <nil> Duration: 1.997060966s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 340.194019ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:28:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 340.224915ms core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 2.456710617s trace: https://jaeger.dev.storj.io/trace/2978a773c720f2e4 FAIL (2025-11-20 17:27:37 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== load=232637]: Uploading piece... Nov 20 17:25:10.582877896 (223h38m35.350561328s) [noise=true , address=81.171.14.70:4029 , piece_id=XJF5EGWPXCL3PXV6WXEL46JAHMHZQGGR4AU5IOUNQLBG4C3BGQUA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.584580409 (223h38m35.352263843s) [noise=true , address=185.167.204.205:4026 , piece_id=YVFA5OFDVPAN3Q7GP7NIO6MC2N47G2MEWLJ7RVJEIP7UBHBYD5NA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.586378791 (223h38m35.354062219s) [noise=true , address=185.167.204.205:4036 , piece_id=LZSPRFBBIJPCW3IWHNW3SNKX4YTW7RCXTOOLE5SBGR2Z4WJHVCZA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.588641711 (223h38m35.356325142s) [noise=true , address=185.167.204.205:4015 , piece_id=ZC3TZGH2N7NHRNMFGG2SVOS5UGTD4CY4OCLCQL2EEIN6RHEMQKIQ, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.591403897 (223h38m35.359087325s) [noise=true , address=185.167.204.205:4034 , piece_id=MPKSGXYASP3YGAXWHL4PLGA7PJX56ELPS6NPJQIKLQN3GOYAUFDA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.593348617 (223h38m35.361032051s) [noise=true , address=81.171.14.70:4031 , piece_id=SHBKH5K4C7T7D7XMAAD6FDA36VCEJXQILDGEVNJZLTMD6EXLBL5A, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.595567737 (223h38m35.363251162s) [noise=true , address=185.167.204.205:4002 , piece_id=27KMJOJSSJTIHEO5ZKY66GIG66P5QAESU4HHHLTI6KV2WZCFRWYQ, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.597504768 (223h38m35.365188202s) [noise=true , address=81.171.14.70:4028 , piece_id=HEZAKOJDOQEBIFDRGYIXSCUWLPKUBTA6MRKQPR6665ZQXEPVIGQA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.599544427 (223h38m35.367227861s) [noise=true , address=81.171.14.70:4038 , piece_id=GCUUKZQY36F6XWCXJGHP6SSK5OKAH47HLPNBC76GVNNQID4DVLDA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.600434395 (223h38m35.36811783s) [noise=true , address=185.167.204.206:4065 , piece_id=UHVHSM2N5H73YGCYEJ26JWO6IZZ76JTTYWM5ZZZZ2HMERRK7VDWA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.992513100 (223h38m35.760196525s) [upload=232637]: Next returned nil segment Nov 20 17:25:10.992541848 (223h38m35.760225277s) [upload=232637]: Waiting for error group managing segments... Nov 20 17:25:11.164544942 (223h38m35.932228366s) [noise=true , address=89.149.201.147:4022 , piece_id=4NQ4J3UJAABVO3SFMRORPXBTUGYXZMJNUWWTAKFIH3ZD5EVF3UXQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.172767990 (223h38m35.940451418s) [noise=true , address=81.171.14.70:4031 , piece_id=SHBKH5K4C7T7D7XMAAD6FDA36VCEJXQILDGEVNJZLTMD6EXLBL5A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.182086404 (223h38m35.949769833s) [noise=true , address=185.167.204.205:4015 , piece_id=ZC3TZGH2N7NHRNMFGG2SVOS5UGTD4CY4OCLCQL2EEIN6RHEMQKIQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.182810476 (223h38m35.950493902s) [noise=true , address=81.171.14.70:4043 , piece_id=6474UQDNSX4UVDXJWZKSDBB5T47HGTYJZMJNTFK3PEQBR2KAY7RA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.183742078 (223h38m35.951425504s) [noise=true , address=185.167.204.206:4053 , piece_id=AIF5XPKJPTGUA7ENKUUTPPOKQCECEKQPPGEMAUOT5QAISCVMO22Q, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.184529255 (223h38m35.952212684s) [noise=true , address=81.171.14.70:4044 , piece_id=RICD4ZMMWVGLS6UTFKKZFUNKE2LI35QQ6VF4FYRMLDUMQDOQCCXA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.185123500 (223h38m35.952806926s) [noise=true , address=185.167.204.206:4066 , piece_id=H52XDJBLWKNNCE25VSBRNUH6BMW3JPBOFGQSEKF4CI3HBKCQSJZQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.186964052 (223h38m35.954647478s) [noise=true , address=185.167.204.205:4028 , piece_id=FVBJ6WVHKSN43OVNMH66KPBREGS2EWGD2EWCVJQX2WVMI27MPC7A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.191410051 (223h38m35.959093477s) [noise=true , address=185.167.204.206:4054 , piece_id=LCBZUECG7VIQOE3X4VT6AT4HEQEEOTHEFBPSRXG7NUJC4PCGQ72Q, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.194616300 (223h38m35.962299725s) [noise=true , address=89.149.201.147:4001 , piece_id=7YTUOPW4BA4KDNJAXDHNUENBDIXNGZZW6576XQD5D3ISQXKBTYTQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.195124666 (223h38m35.962808091s) [noise=true , address=81.171.14.70:4027 , piece_id=OOS4I4CDV2TN42MWMHZBRCQWXGYXLHJTTUONDMTAHSQJKKPROYJA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.196224298 (223h38m35.963907724s) [noise=true , address=89.149.201.147:4005 , piece_id=U64XSFTL2TDVWNNOUQ6ULUGBPNU5XGQ6SI3HU7ZP6PPLZ7PZVRQA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.198703124 (223h38m35.966386601s) [noise=true , address=185.167.204.206:4047 , piece_id=YZFK5OW7TWWIJ6FLA2XMAU3YURPKT2U5FGZYH6YLRNHUUDIV5N7A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.201965022 (223h38m35.969648447s) [noise=true , address=185.167.204.205:4034 , piece_id=MPKSGXYASP3YGAXWHL4PLGA7PJX56ELPS6NPJQIKLQN3GOYAUFDA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.202483607 (223h38m35.970167033s) [noise=true , address=185.167.204.205:4002 , piece_id=27KMJOJSSJTIHEO5ZKY66GIG66P5QAESU4HHHLTI6KV2WZCFRWYQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.211202242 (223h38m35.978885672s) [noise=true , address=185.167.204.205:4036 , piece_id=LZSPRFBBIJPCW3IWHNW3SNKX4YTW7RCXTOOLE5SBGR2Z4WJHVCZA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.216399102 (223h38m35.984082529s) [noise=true , address=185.167.204.206:4046 , piece_id=VY3VEVR5VIHEH3MWJKDYS2UM62EP7ELZU3JI346NDESEORJIPZDQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.226445839 (223h38m35.994129265s) [noise=true , address=185.167.204.205:4026 , piece_id=YVFA5OFDVPAN3Q7GP7NIO6MC2N47G2MEWLJ7RVJEIP7UBHBYD5NA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.240411560 (223h38m36.008094989s) [noise=true , address=185.167.204.206:4065 , piece_id=UHVHSM2N5H73YGCYEJ26JWO6IZZ76JTTYWM5ZZZZ2HMERRK7VDWA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.240958840 (223h38m36.008642266s) [noise=true , address=81.171.14.70:4047 , piece_id=D46YVEDXW2RT6OUYO73PHKREJF55SOCRWX6AMJ6RUWUC7G67426A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.241565965 (223h38m36.009249392s) [noise=true , address=81.171.14.70:4036 , piece_id=RNDJKD3UBSBBCUSY6AMAJ46JA4FPSOOQMWP247A3CGPMMTZXWQUA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.242175176 (223h38m36.009858606s) [noise=true , address=81.171.14.70:4028 , piece_id=HEZAKOJDOQEBIFDRGYIXSCUWLPKUBTA6MRKQPR6665ZQXEPVIGQA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.242769247 (223h38m36.010452683s) [noise=true , address=81.171.14.70:4030 , piece_id=QAOOJJY2QL6NZQ3DN4NG5MNQCLPFBEZ73BJ4ESWPHMMHFJZV3I6Q, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.243245042 (223h38m36.010928472s) [noise=true , address=81.171.14.70:4029 , piece_id=XJF5EGWPXCL3PXV6WXEL46JAHMHZQGGR4AU5IOUNQLBG4C3BGQUA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.247956007 (223h38m36.015639437s) [noise=true , address=81.171.14.70:4048 , piece_id=KUYGLZGDWTEMEG637NPTQCKAEOAU42SCT4QKIGTAFTIVTUTMUV6A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.251384119 (223h38m36.019067547s) [noise=true , address=81.171.14.70:4025 , piece_id=EZNZI3X3CQI5B6ZMKVZABPZ5IZHVVS722EXVLWRHEF625CZFGORA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.251410363 (223h38m36.019093791s) [seg_pos={0 0}, upload=232637]: Segment reached optimal threshold of 26 pieces. Nov 20 17:25:11.251626678 (223h38m36.019310112s) [noise=true , address=89.149.201.147:4018 , piece_id=EF6O3MQ6T6Y6SZLNBTHJGH4U52JOUUIT5FJSRBBZB3R3XRH7MDKA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua): protocol: expected piece hash; context canceled; EOF Nov 20 17:25:11.251642292 (223h38m36.019325718s) [noise=true , address=89.149.201.147:4018 , piece_id=EF6O3MQ6T6Y6SZLNBTHJGH4U52JOUUIT5FJSRBBZB3R3XRH7MDKA, seg_pos={0 0}, upload=232637]: Long tail cancellation due to optimal threshold Nov 20 17:25:11.252013645 (223h38m36.019697078s) [noise=true , address=81.171.14.70:4038 , piece_id=GCUUKZQY36F6XWCXJGHP6SSK5OKAH47HLPNBC76GVNNQID4DVLDA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB): protocol: expected piece hash; context canceled; EOF Nov 20 17:25:11.252022652 (223h38m36.019706079s) [noise=true , address=81.171.14.70:4038 , piece_id=GCUUKZQY36F6XWCXJGHP6SSK5OKAH47HLPNBC76GVNNQID4DVLDA, seg_pos={0 0}, upload=232637]: Long tail cancellation due to optimal threshold Nov 20 17:25:11.252045137 (223h38m36.019728563s) [seg_pos={0 0}, upload=232637]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 20 17:25:11.252321599 (223h38m36.020005025s) [upload=232637]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 20 17:25:11.252330210 (223h38m36.020013635s) [upload=232637]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 17:25:11.439953049 (223h38m36.207636477s) [upload=232637]: Done uploading segments... Nov 20 17:25:11.439975748 (223h38m36.207659185s) [upload=232637]: Upload finished. err: <nil> Duration: 2.45674965s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 301.159857ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:27:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 301.18856ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 302.470866ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:26:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 302.511975ms core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 2.46974739s trace: https://jaeger.dev.storj.io/trace/2978a773c720f2e4 FAIL (2025-11-20 17:26:41 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== load=232637]: Uploading piece... Nov 20 17:25:10.582877896 (223h38m35.350561328s) [noise=true , address=81.171.14.70:4029 , piece_id=XJF5EGWPXCL3PXV6WXEL46JAHMHZQGGR4AU5IOUNQLBG4C3BGQUA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.584580409 (223h38m35.352263843s) [noise=true , address=185.167.204.205:4026 , piece_id=YVFA5OFDVPAN3Q7GP7NIO6MC2N47G2MEWLJ7RVJEIP7UBHBYD5NA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.586378791 (223h38m35.354062219s) [noise=true , address=185.167.204.205:4036 , piece_id=LZSPRFBBIJPCW3IWHNW3SNKX4YTW7RCXTOOLE5SBGR2Z4WJHVCZA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.588641711 (223h38m35.356325142s) [noise=true , address=185.167.204.205:4015 , piece_id=ZC3TZGH2N7NHRNMFGG2SVOS5UGTD4CY4OCLCQL2EEIN6RHEMQKIQ, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.591403897 (223h38m35.359087325s) [noise=true , address=185.167.204.205:4034 , piece_id=MPKSGXYASP3YGAXWHL4PLGA7PJX56ELPS6NPJQIKLQN3GOYAUFDA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.593348617 (223h38m35.361032051s) [noise=true , address=81.171.14.70:4031 , piece_id=SHBKH5K4C7T7D7XMAAD6FDA36VCEJXQILDGEVNJZLTMD6EXLBL5A, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.595567737 (223h38m35.363251162s) [noise=true , address=185.167.204.205:4002 , piece_id=27KMJOJSSJTIHEO5ZKY66GIG66P5QAESU4HHHLTI6KV2WZCFRWYQ, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.597504768 (223h38m35.365188202s) [noise=true , address=81.171.14.70:4028 , piece_id=HEZAKOJDOQEBIFDRGYIXSCUWLPKUBTA6MRKQPR6665ZQXEPVIGQA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.599544427 (223h38m35.367227861s) [noise=true , address=81.171.14.70:4038 , piece_id=GCUUKZQY36F6XWCXJGHP6SSK5OKAH47HLPNBC76GVNNQID4DVLDA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.600434395 (223h38m35.36811783s) [noise=true , address=185.167.204.206:4065 , piece_id=UHVHSM2N5H73YGCYEJ26JWO6IZZ76JTTYWM5ZZZZ2HMERRK7VDWA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.992513100 (223h38m35.760196525s) [upload=232637]: Next returned nil segment Nov 20 17:25:10.992541848 (223h38m35.760225277s) [upload=232637]: Waiting for error group managing segments... Nov 20 17:25:11.164544942 (223h38m35.932228366s) [noise=true , address=89.149.201.147:4022 , piece_id=4NQ4J3UJAABVO3SFMRORPXBTUGYXZMJNUWWTAKFIH3ZD5EVF3UXQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.172767990 (223h38m35.940451418s) [noise=true , address=81.171.14.70:4031 , piece_id=SHBKH5K4C7T7D7XMAAD6FDA36VCEJXQILDGEVNJZLTMD6EXLBL5A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.182086404 (223h38m35.949769833s) [noise=true , address=185.167.204.205:4015 , piece_id=ZC3TZGH2N7NHRNMFGG2SVOS5UGTD4CY4OCLCQL2EEIN6RHEMQKIQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.182810476 (223h38m35.950493902s) [noise=true , address=81.171.14.70:4043 , piece_id=6474UQDNSX4UVDXJWZKSDBB5T47HGTYJZMJNTFK3PEQBR2KAY7RA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.183742078 (223h38m35.951425504s) [noise=true , address=185.167.204.206:4053 , piece_id=AIF5XPKJPTGUA7ENKUUTPPOKQCECEKQPPGEMAUOT5QAISCVMO22Q, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.184529255 (223h38m35.952212684s) [noise=true , address=81.171.14.70:4044 , piece_id=RICD4ZMMWVGLS6UTFKKZFUNKE2LI35QQ6VF4FYRMLDUMQDOQCCXA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.185123500 (223h38m35.952806926s) [noise=true , address=185.167.204.206:4066 , piece_id=H52XDJBLWKNNCE25VSBRNUH6BMW3JPBOFGQSEKF4CI3HBKCQSJZQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.186964052 (223h38m35.954647478s) [noise=true , address=185.167.204.205:4028 , piece_id=FVBJ6WVHKSN43OVNMH66KPBREGS2EWGD2EWCVJQX2WVMI27MPC7A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.191410051 (223h38m35.959093477s) [noise=true , address=185.167.204.206:4054 , piece_id=LCBZUECG7VIQOE3X4VT6AT4HEQEEOTHEFBPSRXG7NUJC4PCGQ72Q, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.194616300 (223h38m35.962299725s) [noise=true , address=89.149.201.147:4001 , piece_id=7YTUOPW4BA4KDNJAXDHNUENBDIXNGZZW6576XQD5D3ISQXKBTYTQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.195124666 (223h38m35.962808091s) [noise=true , address=81.171.14.70:4027 , piece_id=OOS4I4CDV2TN42MWMHZBRCQWXGYXLHJTTUONDMTAHSQJKKPROYJA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.196224298 (223h38m35.963907724s) [noise=true , address=89.149.201.147:4005 , piece_id=U64XSFTL2TDVWNNOUQ6ULUGBPNU5XGQ6SI3HU7ZP6PPLZ7PZVRQA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.198703124 (223h38m35.966386601s) [noise=true , address=185.167.204.206:4047 , piece_id=YZFK5OW7TWWIJ6FLA2XMAU3YURPKT2U5FGZYH6YLRNHUUDIV5N7A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.201965022 (223h38m35.969648447s) [noise=true , address=185.167.204.205:4034 , piece_id=MPKSGXYASP3YGAXWHL4PLGA7PJX56ELPS6NPJQIKLQN3GOYAUFDA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.202483607 (223h38m35.970167033s) [noise=true , address=185.167.204.205:4002 , piece_id=27KMJOJSSJTIHEO5ZKY66GIG66P5QAESU4HHHLTI6KV2WZCFRWYQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.211202242 (223h38m35.978885672s) [noise=true , address=185.167.204.205:4036 , piece_id=LZSPRFBBIJPCW3IWHNW3SNKX4YTW7RCXTOOLE5SBGR2Z4WJHVCZA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.216399102 (223h38m35.984082529s) [noise=true , address=185.167.204.206:4046 , piece_id=VY3VEVR5VIHEH3MWJKDYS2UM62EP7ELZU3JI346NDESEORJIPZDQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.226445839 (223h38m35.994129265s) [noise=true , address=185.167.204.205:4026 , piece_id=YVFA5OFDVPAN3Q7GP7NIO6MC2N47G2MEWLJ7RVJEIP7UBHBYD5NA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.240411560 (223h38m36.008094989s) [noise=true , address=185.167.204.206:4065 , piece_id=UHVHSM2N5H73YGCYEJ26JWO6IZZ76JTTYWM5ZZZZ2HMERRK7VDWA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.240958840 (223h38m36.008642266s) [noise=true , address=81.171.14.70:4047 , piece_id=D46YVEDXW2RT6OUYO73PHKREJF55SOCRWX6AMJ6RUWUC7G67426A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.241565965 (223h38m36.009249392s) [noise=true , address=81.171.14.70:4036 , piece_id=RNDJKD3UBSBBCUSY6AMAJ46JA4FPSOOQMWP247A3CGPMMTZXWQUA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.242175176 (223h38m36.009858606s) [noise=true , address=81.171.14.70:4028 , piece_id=HEZAKOJDOQEBIFDRGYIXSCUWLPKUBTA6MRKQPR6665ZQXEPVIGQA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.242769247 (223h38m36.010452683s) [noise=true , address=81.171.14.70:4030 , piece_id=QAOOJJY2QL6NZQ3DN4NG5MNQCLPFBEZ73BJ4ESWPHMMHFJZV3I6Q, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.243245042 (223h38m36.010928472s) [noise=true , address=81.171.14.70:4029 , piece_id=XJF5EGWPXCL3PXV6WXEL46JAHMHZQGGR4AU5IOUNQLBG4C3BGQUA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.247956007 (223h38m36.015639437s) [noise=true , address=81.171.14.70:4048 , piece_id=KUYGLZGDWTEMEG637NPTQCKAEOAU42SCT4QKIGTAFTIVTUTMUV6A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.251384119 (223h38m36.019067547s) [noise=true , address=81.171.14.70:4025 , piece_id=EZNZI3X3CQI5B6ZMKVZABPZ5IZHVVS722EXVLWRHEF625CZFGORA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.251410363 (223h38m36.019093791s) [seg_pos={0 0}, upload=232637]: Segment reached optimal threshold of 26 pieces. Nov 20 17:25:11.251626678 (223h38m36.019310112s) [noise=true , address=89.149.201.147:4018 , piece_id=EF6O3MQ6T6Y6SZLNBTHJGH4U52JOUUIT5FJSRBBZB3R3XRH7MDKA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua): protocol: expected piece hash; context canceled; EOF Nov 20 17:25:11.251642292 (223h38m36.019325718s) [noise=true , address=89.149.201.147:4018 , piece_id=EF6O3MQ6T6Y6SZLNBTHJGH4U52JOUUIT5FJSRBBZB3R3XRH7MDKA, seg_pos={0 0}, upload=232637]: Long tail cancellation due to optimal threshold Nov 20 17:25:11.252013645 (223h38m36.019697078s) [noise=true , address=81.171.14.70:4038 , piece_id=GCUUKZQY36F6XWCXJGHP6SSK5OKAH47HLPNBC76GVNNQID4DVLDA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB): protocol: expected piece hash; context canceled; EOF Nov 20 17:25:11.252022652 (223h38m36.019706079s) [noise=true , address=81.171.14.70:4038 , piece_id=GCUUKZQY36F6XWCXJGHP6SSK5OKAH47HLPNBC76GVNNQID4DVLDA, seg_pos={0 0}, upload=232637]: Long tail cancellation due to optimal threshold Nov 20 17:25:11.252045137 (223h38m36.019728563s) [seg_pos={0 0}, upload=232637]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 20 17:25:11.252321599 (223h38m36.020005025s) [upload=232637]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 20 17:25:11.252330210 (223h38m36.020013635s) [upload=232637]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 17:25:11.439953049 (223h38m36.207636477s) [upload=232637]: Done uploading segments... Nov 20 17:25:11.439975748 (223h38m36.207659185s) [upload=232637]: Upload finished. err: <nil> Duration: 2.469773142s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 2.541457818s trace: https://jaeger.dev.storj.io/trace/2978a773c720f2e4 FAIL (2025-11-20 17:26:12 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== load=232637]: Uploading piece... Nov 20 17:25:10.582877896 (223h38m35.350561328s) [noise=true , address=81.171.14.70:4029 , piece_id=XJF5EGWPXCL3PXV6WXEL46JAHMHZQGGR4AU5IOUNQLBG4C3BGQUA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.584580409 (223h38m35.352263843s) [noise=true , address=185.167.204.205:4026 , piece_id=YVFA5OFDVPAN3Q7GP7NIO6MC2N47G2MEWLJ7RVJEIP7UBHBYD5NA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.586378791 (223h38m35.354062219s) [noise=true , address=185.167.204.205:4036 , piece_id=LZSPRFBBIJPCW3IWHNW3SNKX4YTW7RCXTOOLE5SBGR2Z4WJHVCZA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.588641711 (223h38m35.356325142s) [noise=true , address=185.167.204.205:4015 , piece_id=ZC3TZGH2N7NHRNMFGG2SVOS5UGTD4CY4OCLCQL2EEIN6RHEMQKIQ, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.591403897 (223h38m35.359087325s) [noise=true , address=185.167.204.205:4034 , piece_id=MPKSGXYASP3YGAXWHL4PLGA7PJX56ELPS6NPJQIKLQN3GOYAUFDA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.593348617 (223h38m35.361032051s) [noise=true , address=81.171.14.70:4031 , piece_id=SHBKH5K4C7T7D7XMAAD6FDA36VCEJXQILDGEVNJZLTMD6EXLBL5A, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.595567737 (223h38m35.363251162s) [noise=true , address=185.167.204.205:4002 , piece_id=27KMJOJSSJTIHEO5ZKY66GIG66P5QAESU4HHHLTI6KV2WZCFRWYQ, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.597504768 (223h38m35.365188202s) [noise=true , address=81.171.14.70:4028 , piece_id=HEZAKOJDOQEBIFDRGYIXSCUWLPKUBTA6MRKQPR6665ZQXEPVIGQA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.599544427 (223h38m35.367227861s) [noise=true , address=81.171.14.70:4038 , piece_id=GCUUKZQY36F6XWCXJGHP6SSK5OKAH47HLPNBC76GVNNQID4DVLDA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.600434395 (223h38m35.36811783s) [noise=true , address=185.167.204.206:4065 , piece_id=UHVHSM2N5H73YGCYEJ26JWO6IZZ76JTTYWM5ZZZZ2HMERRK7VDWA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.992513100 (223h38m35.760196525s) [upload=232637]: Next returned nil segment Nov 20 17:25:10.992541848 (223h38m35.760225277s) [upload=232637]: Waiting for error group managing segments... Nov 20 17:25:11.164544942 (223h38m35.932228366s) [noise=true , address=89.149.201.147:4022 , piece_id=4NQ4J3UJAABVO3SFMRORPXBTUGYXZMJNUWWTAKFIH3ZD5EVF3UXQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.172767990 (223h38m35.940451418s) [noise=true , address=81.171.14.70:4031 , piece_id=SHBKH5K4C7T7D7XMAAD6FDA36VCEJXQILDGEVNJZLTMD6EXLBL5A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.182086404 (223h38m35.949769833s) [noise=true , address=185.167.204.205:4015 , piece_id=ZC3TZGH2N7NHRNMFGG2SVOS5UGTD4CY4OCLCQL2EEIN6RHEMQKIQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.182810476 (223h38m35.950493902s) [noise=true , address=81.171.14.70:4043 , piece_id=6474UQDNSX4UVDXJWZKSDBB5T47HGTYJZMJNTFK3PEQBR2KAY7RA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.183742078 (223h38m35.951425504s) [noise=true , address=185.167.204.206:4053 , piece_id=AIF5XPKJPTGUA7ENKUUTPPOKQCECEKQPPGEMAUOT5QAISCVMO22Q, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.184529255 (223h38m35.952212684s) [noise=true , address=81.171.14.70:4044 , piece_id=RICD4ZMMWVGLS6UTFKKZFUNKE2LI35QQ6VF4FYRMLDUMQDOQCCXA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.185123500 (223h38m35.952806926s) [noise=true , address=185.167.204.206:4066 , piece_id=H52XDJBLWKNNCE25VSBRNUH6BMW3JPBOFGQSEKF4CI3HBKCQSJZQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.186964052 (223h38m35.954647478s) [noise=true , address=185.167.204.205:4028 , piece_id=FVBJ6WVHKSN43OVNMH66KPBREGS2EWGD2EWCVJQX2WVMI27MPC7A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.191410051 (223h38m35.959093477s) [noise=true , address=185.167.204.206:4054 , piece_id=LCBZUECG7VIQOE3X4VT6AT4HEQEEOTHEFBPSRXG7NUJC4PCGQ72Q, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.194616300 (223h38m35.962299725s) [noise=true , address=89.149.201.147:4001 , piece_id=7YTUOPW4BA4KDNJAXDHNUENBDIXNGZZW6576XQD5D3ISQXKBTYTQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.195124666 (223h38m35.962808091s) [noise=true , address=81.171.14.70:4027 , piece_id=OOS4I4CDV2TN42MWMHZBRCQWXGYXLHJTTUONDMTAHSQJKKPROYJA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.196224298 (223h38m35.963907724s) [noise=true , address=89.149.201.147:4005 , piece_id=U64XSFTL2TDVWNNOUQ6ULUGBPNU5XGQ6SI3HU7ZP6PPLZ7PZVRQA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.198703124 (223h38m35.966386601s) [noise=true , address=185.167.204.206:4047 , piece_id=YZFK5OW7TWWIJ6FLA2XMAU3YURPKT2U5FGZYH6YLRNHUUDIV5N7A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.201965022 (223h38m35.969648447s) [noise=true , address=185.167.204.205:4034 , piece_id=MPKSGXYASP3YGAXWHL4PLGA7PJX56ELPS6NPJQIKLQN3GOYAUFDA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.202483607 (223h38m35.970167033s) [noise=true , address=185.167.204.205:4002 , piece_id=27KMJOJSSJTIHEO5ZKY66GIG66P5QAESU4HHHLTI6KV2WZCFRWYQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.211202242 (223h38m35.978885672s) [noise=true , address=185.167.204.205:4036 , piece_id=LZSPRFBBIJPCW3IWHNW3SNKX4YTW7RCXTOOLE5SBGR2Z4WJHVCZA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.216399102 (223h38m35.984082529s) [noise=true , address=185.167.204.206:4046 , piece_id=VY3VEVR5VIHEH3MWJKDYS2UM62EP7ELZU3JI346NDESEORJIPZDQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.226445839 (223h38m35.994129265s) [noise=true , address=185.167.204.205:4026 , piece_id=YVFA5OFDVPAN3Q7GP7NIO6MC2N47G2MEWLJ7RVJEIP7UBHBYD5NA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.240411560 (223h38m36.008094989s) [noise=true , address=185.167.204.206:4065 , piece_id=UHVHSM2N5H73YGCYEJ26JWO6IZZ76JTTYWM5ZZZZ2HMERRK7VDWA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.240958840 (223h38m36.008642266s) [noise=true , address=81.171.14.70:4047 , piece_id=D46YVEDXW2RT6OUYO73PHKREJF55SOCRWX6AMJ6RUWUC7G67426A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.241565965 (223h38m36.009249392s) [noise=true , address=81.171.14.70:4036 , piece_id=RNDJKD3UBSBBCUSY6AMAJ46JA4FPSOOQMWP247A3CGPMMTZXWQUA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.242175176 (223h38m36.009858606s) [noise=true , address=81.171.14.70:4028 , piece_id=HEZAKOJDOQEBIFDRGYIXSCUWLPKUBTA6MRKQPR6665ZQXEPVIGQA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.242769247 (223h38m36.010452683s) [noise=true , address=81.171.14.70:4030 , piece_id=QAOOJJY2QL6NZQ3DN4NG5MNQCLPFBEZ73BJ4ESWPHMMHFJZV3I6Q, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.243245042 (223h38m36.010928472s) [noise=true , address=81.171.14.70:4029 , piece_id=XJF5EGWPXCL3PXV6WXEL46JAHMHZQGGR4AU5IOUNQLBG4C3BGQUA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.247956007 (223h38m36.015639437s) [noise=true , address=81.171.14.70:4048 , piece_id=KUYGLZGDWTEMEG637NPTQCKAEOAU42SCT4QKIGTAFTIVTUTMUV6A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.251384119 (223h38m36.019067547s) [noise=true , address=81.171.14.70:4025 , piece_id=EZNZI3X3CQI5B6ZMKVZABPZ5IZHVVS722EXVLWRHEF625CZFGORA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.251410363 (223h38m36.019093791s) [seg_pos={0 0}, upload=232637]: Segment reached optimal threshold of 26 pieces. Nov 20 17:25:11.251626678 (223h38m36.019310112s) [noise=true , address=89.149.201.147:4018 , piece_id=EF6O3MQ6T6Y6SZLNBTHJGH4U52JOUUIT5FJSRBBZB3R3XRH7MDKA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua): protocol: expected piece hash; context canceled; EOF Nov 20 17:25:11.251642292 (223h38m36.019325718s) [noise=true , address=89.149.201.147:4018 , piece_id=EF6O3MQ6T6Y6SZLNBTHJGH4U52JOUUIT5FJSRBBZB3R3XRH7MDKA, seg_pos={0 0}, upload=232637]: Long tail cancellation due to optimal threshold Nov 20 17:25:11.252013645 (223h38m36.019697078s) [noise=true , address=81.171.14.70:4038 , piece_id=GCUUKZQY36F6XWCXJGHP6SSK5OKAH47HLPNBC76GVNNQID4DVLDA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB): protocol: expected piece hash; context canceled; EOF Nov 20 17:25:11.252022652 (223h38m36.019706079s) [noise=true , address=81.171.14.70:4038 , piece_id=GCUUKZQY36F6XWCXJGHP6SSK5OKAH47HLPNBC76GVNNQID4DVLDA, seg_pos={0 0}, upload=232637]: Long tail cancellation due to optimal threshold Nov 20 17:25:11.252045137 (223h38m36.019728563s) [seg_pos={0 0}, upload=232637]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 20 17:25:11.252321599 (223h38m36.020005025s) [upload=232637]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 20 17:25:11.252330210 (223h38m36.020013635s) [upload=232637]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 17:25:11.439953049 (223h38m36.207636477s) [upload=232637]: Done uploading segments... Nov 20 17:25:11.439975748 (223h38m36.207659185s) [upload=232637]: Upload finished. err: <nil> Duration: 2.541478684s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.004055798s trace: https://jaeger.dev.storj.io/trace/2978a773c720f2e4 FAIL (2025-11-20 17:26:10 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== load=232637]: Uploading piece... Nov 20 17:25:10.582877896 (223h38m35.350561328s) [noise=true , address=81.171.14.70:4029 , piece_id=XJF5EGWPXCL3PXV6WXEL46JAHMHZQGGR4AU5IOUNQLBG4C3BGQUA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.584580409 (223h38m35.352263843s) [noise=true , address=185.167.204.205:4026 , piece_id=YVFA5OFDVPAN3Q7GP7NIO6MC2N47G2MEWLJ7RVJEIP7UBHBYD5NA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.586378791 (223h38m35.354062219s) [noise=true , address=185.167.204.205:4036 , piece_id=LZSPRFBBIJPCW3IWHNW3SNKX4YTW7RCXTOOLE5SBGR2Z4WJHVCZA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.588641711 (223h38m35.356325142s) [noise=true , address=185.167.204.205:4015 , piece_id=ZC3TZGH2N7NHRNMFGG2SVOS5UGTD4CY4OCLCQL2EEIN6RHEMQKIQ, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.591403897 (223h38m35.359087325s) [noise=true , address=185.167.204.205:4034 , piece_id=MPKSGXYASP3YGAXWHL4PLGA7PJX56ELPS6NPJQIKLQN3GOYAUFDA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.593348617 (223h38m35.361032051s) [noise=true , address=81.171.14.70:4031 , piece_id=SHBKH5K4C7T7D7XMAAD6FDA36VCEJXQILDGEVNJZLTMD6EXLBL5A, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.595567737 (223h38m35.363251162s) [noise=true , address=185.167.204.205:4002 , piece_id=27KMJOJSSJTIHEO5ZKY66GIG66P5QAESU4HHHLTI6KV2WZCFRWYQ, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.597504768 (223h38m35.365188202s) [noise=true , address=81.171.14.70:4028 , piece_id=HEZAKOJDOQEBIFDRGYIXSCUWLPKUBTA6MRKQPR6665ZQXEPVIGQA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.599544427 (223h38m35.367227861s) [noise=true , address=81.171.14.70:4038 , piece_id=GCUUKZQY36F6XWCXJGHP6SSK5OKAH47HLPNBC76GVNNQID4DVLDA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.600434395 (223h38m35.36811783s) [noise=true , address=185.167.204.206:4065 , piece_id=UHVHSM2N5H73YGCYEJ26JWO6IZZ76JTTYWM5ZZZZ2HMERRK7VDWA, seg_pos={0 0}, upload=232637]: Uploading piece... Nov 20 17:25:10.992513100 (223h38m35.760196525s) [upload=232637]: Next returned nil segment Nov 20 17:25:10.992541848 (223h38m35.760225277s) [upload=232637]: Waiting for error group managing segments... Nov 20 17:25:11.164544942 (223h38m35.932228366s) [noise=true , address=89.149.201.147:4022 , piece_id=4NQ4J3UJAABVO3SFMRORPXBTUGYXZMJNUWWTAKFIH3ZD5EVF3UXQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.172767990 (223h38m35.940451418s) [noise=true , address=81.171.14.70:4031 , piece_id=SHBKH5K4C7T7D7XMAAD6FDA36VCEJXQILDGEVNJZLTMD6EXLBL5A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.182086404 (223h38m35.949769833s) [noise=true , address=185.167.204.205:4015 , piece_id=ZC3TZGH2N7NHRNMFGG2SVOS5UGTD4CY4OCLCQL2EEIN6RHEMQKIQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.182810476 (223h38m35.950493902s) [noise=true , address=81.171.14.70:4043 , piece_id=6474UQDNSX4UVDXJWZKSDBB5T47HGTYJZMJNTFK3PEQBR2KAY7RA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.183742078 (223h38m35.951425504s) [noise=true , address=185.167.204.206:4053 , piece_id=AIF5XPKJPTGUA7ENKUUTPPOKQCECEKQPPGEMAUOT5QAISCVMO22Q, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.184529255 (223h38m35.952212684s) [noise=true , address=81.171.14.70:4044 , piece_id=RICD4ZMMWVGLS6UTFKKZFUNKE2LI35QQ6VF4FYRMLDUMQDOQCCXA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.185123500 (223h38m35.952806926s) [noise=true , address=185.167.204.206:4066 , piece_id=H52XDJBLWKNNCE25VSBRNUH6BMW3JPBOFGQSEKF4CI3HBKCQSJZQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.186964052 (223h38m35.954647478s) [noise=true , address=185.167.204.205:4028 , piece_id=FVBJ6WVHKSN43OVNMH66KPBREGS2EWGD2EWCVJQX2WVMI27MPC7A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.191410051 (223h38m35.959093477s) [noise=true , address=185.167.204.206:4054 , piece_id=LCBZUECG7VIQOE3X4VT6AT4HEQEEOTHEFBPSRXG7NUJC4PCGQ72Q, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.194616300 (223h38m35.962299725s) [noise=true , address=89.149.201.147:4001 , piece_id=7YTUOPW4BA4KDNJAXDHNUENBDIXNGZZW6576XQD5D3ISQXKBTYTQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.195124666 (223h38m35.962808091s) [noise=true , address=81.171.14.70:4027 , piece_id=OOS4I4CDV2TN42MWMHZBRCQWXGYXLHJTTUONDMTAHSQJKKPROYJA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.196224298 (223h38m35.963907724s) [noise=true , address=89.149.201.147:4005 , piece_id=U64XSFTL2TDVWNNOUQ6ULUGBPNU5XGQ6SI3HU7ZP6PPLZ7PZVRQA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.198703124 (223h38m35.966386601s) [noise=true , address=185.167.204.206:4047 , piece_id=YZFK5OW7TWWIJ6FLA2XMAU3YURPKT2U5FGZYH6YLRNHUUDIV5N7A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.201965022 (223h38m35.969648447s) [noise=true , address=185.167.204.205:4034 , piece_id=MPKSGXYASP3YGAXWHL4PLGA7PJX56ELPS6NPJQIKLQN3GOYAUFDA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.202483607 (223h38m35.970167033s) [noise=true , address=185.167.204.205:4002 , piece_id=27KMJOJSSJTIHEO5ZKY66GIG66P5QAESU4HHHLTI6KV2WZCFRWYQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.211202242 (223h38m35.978885672s) [noise=true , address=185.167.204.205:4036 , piece_id=LZSPRFBBIJPCW3IWHNW3SNKX4YTW7RCXTOOLE5SBGR2Z4WJHVCZA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.216399102 (223h38m35.984082529s) [noise=true , address=185.167.204.206:4046 , piece_id=VY3VEVR5VIHEH3MWJKDYS2UM62EP7ELZU3JI346NDESEORJIPZDQ, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.226445839 (223h38m35.994129265s) [noise=true , address=185.167.204.205:4026 , piece_id=YVFA5OFDVPAN3Q7GP7NIO6MC2N47G2MEWLJ7RVJEIP7UBHBYD5NA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.240411560 (223h38m36.008094989s) [noise=true , address=185.167.204.206:4065 , piece_id=UHVHSM2N5H73YGCYEJ26JWO6IZZ76JTTYWM5ZZZZ2HMERRK7VDWA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.240958840 (223h38m36.008642266s) [noise=true , address=81.171.14.70:4047 , piece_id=D46YVEDXW2RT6OUYO73PHKREJF55SOCRWX6AMJ6RUWUC7G67426A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.241565965 (223h38m36.009249392s) [noise=true , address=81.171.14.70:4036 , piece_id=RNDJKD3UBSBBCUSY6AMAJ46JA4FPSOOQMWP247A3CGPMMTZXWQUA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.242175176 (223h38m36.009858606s) [noise=true , address=81.171.14.70:4028 , piece_id=HEZAKOJDOQEBIFDRGYIXSCUWLPKUBTA6MRKQPR6665ZQXEPVIGQA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.242769247 (223h38m36.010452683s) [noise=true , address=81.171.14.70:4030 , piece_id=QAOOJJY2QL6NZQ3DN4NG5MNQCLPFBEZ73BJ4ESWPHMMHFJZV3I6Q, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.243245042 (223h38m36.010928472s) [noise=true , address=81.171.14.70:4029 , piece_id=XJF5EGWPXCL3PXV6WXEL46JAHMHZQGGR4AU5IOUNQLBG4C3BGQUA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.247956007 (223h38m36.015639437s) [noise=true , address=81.171.14.70:4048 , piece_id=KUYGLZGDWTEMEG637NPTQCKAEOAU42SCT4QKIGTAFTIVTUTMUV6A, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.251384119 (223h38m36.019067547s) [noise=true , address=81.171.14.70:4025 , piece_id=EZNZI3X3CQI5B6ZMKVZABPZ5IZHVVS722EXVLWRHEF625CZFGORA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: <nil> Nov 20 17:25:11.251410363 (223h38m36.019093791s) [seg_pos={0 0}, upload=232637]: Segment reached optimal threshold of 26 pieces. Nov 20 17:25:11.251626678 (223h38m36.019310112s) [noise=true , address=89.149.201.147:4018 , piece_id=EF6O3MQ6T6Y6SZLNBTHJGH4U52JOUUIT5FJSRBBZB3R3XRH7MDKA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua): protocol: expected piece hash; context canceled; EOF Nov 20 17:25:11.251642292 (223h38m36.019325718s) [noise=true , address=89.149.201.147:4018 , piece_id=EF6O3MQ6T6Y6SZLNBTHJGH4U52JOUUIT5FJSRBBZB3R3XRH7MDKA, seg_pos={0 0}, upload=232637]: Long tail cancellation due to optimal threshold Nov 20 17:25:11.252013645 (223h38m36.019697078s) [noise=true , address=81.171.14.70:4038 , piece_id=GCUUKZQY36F6XWCXJGHP6SSK5OKAH47HLPNBC76GVNNQID4DVLDA, seg_pos={0 0}, upload=232637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB): protocol: expected piece hash; context canceled; EOF Nov 20 17:25:11.252022652 (223h38m36.019706079s) [noise=true , address=81.171.14.70:4038 , piece_id=GCUUKZQY36F6XWCXJGHP6SSK5OKAH47HLPNBC76GVNNQID4DVLDA, seg_pos={0 0}, upload=232637]: Long tail cancellation due to optimal threshold Nov 20 17:25:11.252045137 (223h38m36.019728563s) [seg_pos={0 0}, upload=232637]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 20 17:25:11.252321599 (223h38m36.020005025s) [upload=232637]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 20 17:25:11.252330210 (223h38m36.020013635s) [upload=232637]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 17:25:11.439953049 (223h38m36.207636477s) [upload=232637]: Done uploading segments... Nov 20 17:25:11.439975748 (223h38m36.207659185s) [upload=232637]: Upload finished. err: <nil> Duration: 1m0.004080262s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 253.805669ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:25:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 253.876017ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 442.658469ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:25:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 442.672931ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 289.320021ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:24:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 289.350088ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 256.858617ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:23:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 256.88809ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 325.093455ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:22:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 325.121999ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 238.971529ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:22:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 238.990654ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 316.757053ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:21:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 316.78063ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 327.735999ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:21:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 327.761891ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 293.123654ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:21:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 293.167945ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 345.937468ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:20:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 345.965405ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 374.203772ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:19:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 374.221804ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 269.208405ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:19:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 269.240103ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 328.78001ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:18:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 328.82111ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 309.728782ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:17:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 309.760383ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 284.411351ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:16:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 284.452226ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 291.19908ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:16:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 291.222392ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 282.93912ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:14:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 283.030687ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 289.265141ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:14:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 289.294126ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 317.267961ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:13:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 317.289598ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 343.036243ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:12:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 343.070731ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 280.098852ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:11:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 280.123361ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 251.312289ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:11:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 251.356176ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 325.273146ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:10:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 325.30019ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 314.056338ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:09:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 314.085137ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 244.085895ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:08:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 244.111963ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 319.851631ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:07:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 319.877007ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 248.214829ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:07:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 248.240332ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 313.087293ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:06:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 313.134013ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 331.623817ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:06:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 331.647485ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 316.247401ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:05:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 316.275051ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 326.716929ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:05:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 326.767708ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 275.451404ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:04:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 275.473297ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 348.594272ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:03:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 348.612722ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 300.039125ms trace: https://jaeger.dev.storj.io/trace/4c3f0bf095c1ee35 FAIL (2025-11-20 17:02:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 20 16:59:28.751814362 (223h12m53.519497788s) [upload=232198]: Starting upload Nov 20 16:59:28.751887267 (223h12m53.519570685s) [upload=232198]: Done starting upload Nov 20 16:59:28.751969314 (223h12m53.519652729s) [upload=232198]: Uploading segments... Nov 20 16:59:28.752027711 (223h12m53.519711145s) [upload=232198]: Got next segment. Inline: true Nov 20 16:59:28.752032755 (223h12m53.51971618s) [upload=232198]: Waiting for error group managing segments... Nov 20 16:59:28.752093514 (223h12m53.519776939s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 20 16:59:28.752098565 (223h12m53.51978199s) [upload=232198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Nov 20 16:59:28.752101755 (223h12m53.519785179s) [upload=232198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 20 16:59:58.720156033 (223h13m23.487839467s) [upload=232198]: Done uploading segments... Nov 20 16:59:58.720221444 (223h13m23.487904872s) [upload=232198]: Upload finished. err: metaclient: context deadline exceeded Duration:* 300.064951ms core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)