Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 1.54597173s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:42:09 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.546068703s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 1.318535439s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:41:29 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.318560608s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 1.248789988s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:40:22 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.248822862s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 935.716972ms trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:39:22 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 935.832719ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 1.225198935s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:38:56 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.225226018s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 1.220517908s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:38:21 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.220609676s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 1.263991165s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:38:00 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.26401969s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 1.333540927s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:37:28 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.333675193s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 1.329876198s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:36:27 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.32990625s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 1.275896081s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:35:28 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.27592254s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 1.291373064s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:34:43 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.291471015s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 1.089429922s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:34:23 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.089519352s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 1.316583768s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:33:54 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.316608068s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 1.263551201s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:32:42 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.263654783s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 1.19889528s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:31:52 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.198927035s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 1.268947585s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:31:08 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.268976316s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 1.240411285s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:30:12 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.240441611s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 1.222120124s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:29:00 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.222150821s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 1.224228064s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:28:36 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.224255302s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 1.047696106s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:27:23 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.047724994s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 1.199469364s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:26:35 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.199526768s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 1.091454194s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:26:17 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.091495234s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 1.142015936s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:25:44 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.142047008s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 1.283683874s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:25:28 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.283723295s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 1.233544523s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:24:26 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.233576584s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 1.328595453s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:23:31 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.328630179s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 1.392072889s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:23:16 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.392094846s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 1.201814552s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:22:53 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.201854731s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 1.078638694s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:21:52 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.078670775s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 1.354970542s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:21:15 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.355027212s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 1.296773875s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:20:13 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.296823733s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 1.24079127s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:19:33 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.240944814s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 1.203476s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:18:56 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.203508071s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 1.292875242s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:18:05 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.292900107s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 1.201538319s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:17:49 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.201566709s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 1.255353358s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:17:31 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.255389084s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 1.191200556s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:16:57 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.191233928s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 1.249382278s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:16:30 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.24943273s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 1.085157336s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:15:56 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.085187602s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 1.342729277s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:14:56 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.342757398s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 1.102408304s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:14:41 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.102437646s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 1.253672999s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:14:12 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.25379025s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 1.219091927s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:13:22 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.219122773s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 1.350677152s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:12:39 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.350773345s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 1.328772306s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:11:57 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.328838388s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 1.335949365s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:11:04 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1.335970595s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.003547879s trace: https://jaeger.dev.storj.io/trace/74532d121d1f43a FAIL (2026-03-25 01:11:02 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 25 01:10:02.739593972 (446h10m0.336365001s) [upload=499896]: Starting upload Mar 25 01:10:02.739664084 (446h10m0.336435114s) [upload=499896]: Done starting upload Mar 25 01:10:02.740064220 (446h10m0.336835237s) [upload=499896]: Uploading segments... Mar 25 01:10:02.740114755 (446h10m0.336885786s) [upload=499896]: Got next segment. Inline: false Mar 25 01:10:02.740154246 (446h10m0.336925269s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 25 01:10:02.740160433 (446h10m0.336931458s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 25 01:10:02.777356967 (446h10m0.37412799s) [seg_pos={0 0}, upload=499896]: Begin upload segment... Mar 25 01:10:02.777454240 (446h10m0.374225271s) [seg_pos={0 0}, upload=499896]: Done begin upload segment. Mar 25 01:10:02.777518274 (446h10m0.374289305s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778127230 (446h10m0.374898264s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.778549279 (446h10m0.375320315s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779144173 (446h10m0.375915207s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.779653561 (446h10m0.376424587s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780067377 (446h10m0.376838414s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.780149826 (446h10m0.376920861s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781018501 (446h10m0.377789532s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781040080 (446h10m0.377811105s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781140320 (446h10m0.377911347s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.781856321 (446h10m0.378627355s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.782918588 (446h10m0.379689702s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.783828227 (446h10m0.380599261s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.784529361 (446h10m0.381300399s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.785176823 (446h10m0.381947852s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.786463988 (446h10m0.38323502s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.787461700 (446h10m0.38423271s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.799948329 (446h10m0.396719446s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Uploading piece... Mar 25 01:10:02.877196937 (446h10m0.473967975s) [upload=499896]: Next returned nil segment Mar 25 01:10:02.877216178 (446h10m0.473987213s) [upload=499896]: Waiting for error group managing segments... Mar 25 01:10:02.882818211 (446h10m0.479589245s) [noise=true , address=45.91.116.12:4247 , piece_id=IERPGFOFRHXZ2CFAFC25ETHBZMXY74H2USNSOIBB3H3PUO6UVPEA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.884581792 (446h10m0.481352836s) [noise=true , address=77.90.31.19:4859 , piece_id=OE44XQ6BXRACRD4D2FJXM5L6FV2OPYSELUNAZMHQFKKYJZXBZXMQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.887139526 (446h10m0.483910553s) [noise=true , address=77.90.31.17:4797 , piece_id=VML4EWC6FSCJFOHZL75YNHHLU5KGSNUPA4MY4PQ2VHH4SOFTVFIA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.901480332 (446h10m0.498251363s) [noise=true , address=45.91.116.9:4290 , piece_id=C4X62B3CIEMXCPZPH6OQEZVBLT2UZEIK7XIFGKIKHOXYLFIDX2LQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:02.913324991 (446h10m0.510096021s) [noise=true , address=77.90.31.10:4529 , piece_id=L2KPWPA2RLFQKFIDRR4SZTDCJ3JW566HJXF4R2CYTSY4RJAT5J2A, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.249573668 (446h10m0.846344715s) [noise=true , address=167.17.72.70:4766 , piece_id=D56WH4KS7QIPCLPD2IENE7QK5ARSLL2TQGZCD4KJGCHGUQJGYOQQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.263243251 (446h10m0.860014283s) [noise=true , address=167.17.72.71:4786 , piece_id=6I3V7NMHIYMO2EBEYKSWJO5QAIXHOTEZV7WKTFXWHSBDBRS2CAXQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.278053339 (446h10m0.874824367s) [noise=true , address=167.17.72.68:4653 , piece_id=C6JYFO6OK6UHQJ2V5NTS5H6WB7TACWK6VHDJP64KYD3KJFR7BZHA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406193705 (446h10m1.00296472s) [noise=true , address=193.149.164.67:4009 , piece_id=5P56OXZQL6FKWKKZMIZPABYKILTX4GPJMNZHTD5OWADQPBLZOU5Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.406788254 (446h10m1.003559267s) [noise=true , address=193.149.164.66:4978 , piece_id=5FDWAZYHYQ4DKT5RK3ED6NSVWOPDNUL3LGK5MY3ZSDAE74GFZNAQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.419880852 (446h10m1.016651875s) [noise=true , address=193.149.164.68:4060 , piece_id=JPTBGLLUWY42ZYSHJCAHBDKZFCGOOGSL2JXLYYXISQ4BKPDFPW6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.447365946 (446h10m1.044136981s) [noise=true , address=193.149.164.70:4118 , piece_id=AI3LCNGKD2B4KT2ID6G2KDZ2GIWWC6JXYQVJXQIEUXHDRXUKAS6Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.450187661 (446h10m1.046958691s) [noise=true , address=193.149.164.143:4694 , piece_id=EXWV6VUQ6BHS7BH4NIBF47JQRDVSJSCHECF45S6QI6XE5Z3A3ZDA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470183493 (446h10m1.066954527s) [noise=true , address=193.149.164.71:4298 , piece_id=SZVECELMX4Q53BLV5KIANGEVPJ5Z7SGUY4PLDTM6PXOFT2DEEVZA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: <nil> Mar 25 01:10:03.470231712 (446h10m1.067002799s) [seg_pos={0 0}, upload=499896]: Satisfied cohort requirements after 14 pieces Mar 25 01:10:03.470573843 (446h10m1.06734488s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470601833 (446h10m1.06737286s) [noise=true , address=145.79.227.43:4164 , piece_id=O4DY6BAHYUPRRXMW3WK75A75UBOWXCD23HEKQE4TKNLFPXY4RCSA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470824195 (446h10m1.067595228s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470839957 (446h10m1.067610988s) [noise=true , address=145.79.227.36:4532 , piece_id=CGSAITEZ5VQV6B22W34GJJCJEGC4TFMPIBB6HHSFGLMKUNHRHYIQ, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.470862841 (446h10m1.067633854s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.470872939 (446h10m1.067643962s) [noise=true , address=64.49.13.140:4510 , piece_id=YO3Y2BKH35FD4OXL5BJOTX4KKHB7KVLNYOKG3OLNMBIDYQO7OKQA, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471128947 (446h10m1.067899982s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CiiMtdAoMhPZwSqwraaQcrrx4vZw6L74vadw712VQbRLpNzKN): protocol: expected piece hash; context canceled; EOF Mar 25 01:10:03.471145761 (446h10m1.067916879s) [noise=true , address=64.49.13.138:4489 , piece_id=QOCFQB2X3WKRPZYYAJ7ZVT7VFAWKOXNECOLNN7C346DKNMQM654Q, seg_pos={0 0}, upload=499896]: Long tail cancellation due to optimal threshold Mar 25 01:10:03.471205622 (446h10m1.067976647s) [seg_pos={0 0}, upload=499896]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Mar 25 01:10:03.471329462 (446h10m1.068100494s) [upload=499896]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 25 01:10:03.471335761 (446h10m1.06810679s) [upload=499896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 25 01:10:03.538370910 (446h10m1.135141944s) [upload=499896]: Done uploading segments... Mar 25 01:10:03.538544157 (446h10m1.135315212s) [upload=499896]: Upload finished. err: <nil> Duration: 1m0.003639835s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 1.202406276s trace: https://jaeger.dev.storj.io/trace/5208d152c65127fc FAIL (2026-03-25 00:44:29 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 1.202435192s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 195.060007ms trace: https://jaeger.dev.storj.io/trace/2183df40fef8c865 FAIL (2026-03-25 00:44:18 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 195.157548ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 226.923252ms trace: https://jaeger.dev.storj.io/trace/2183df40fef8c865 FAIL (2026-03-25 00:43:50 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 226.949471ms core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)