Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 415.111787ms FAIL (2025-07-02 04:49:38 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 415.142103ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 402.221396ms FAIL (2025-07-02 04:49:11 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 402.251754ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 498.81698ms FAIL (2025-07-02 04:48:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 498.892924ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 426.310484ms FAIL (2025-07-02 04:48:13 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 426.339324ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 628.01263ms FAIL (2025-07-02 04:47:08 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 628.036358ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 491.522348ms FAIL (2025-07-02 04:46:13 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 491.550184ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 418.774583ms FAIL (2025-07-02 04:45:22 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 418.807292ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 418.689495ms FAIL (2025-07-02 04:44:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 418.718739ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 423.375864ms FAIL (2025-07-02 04:43:51 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 423.45609ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 488.343852ms FAIL (2025-07-02 04:43:01 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 488.377685ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 449.359875ms FAIL (2025-07-02 04:42:06 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 449.396222ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 584.295781ms FAIL (2025-07-02 04:41:05 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 584.325354ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 3.013682862s FAIL (2025-07-02 04:40:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 3.01373898s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 605.377153ms FAIL (2025-07-02 04:40:16 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 605.400779ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 483.294468ms FAIL (2025-07-02 04:39:16 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 483.322104ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 480.695436ms FAIL (2025-07-02 04:38:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 480.727158ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 448.879458ms FAIL (2025-07-02 04:37:27 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 448.911607ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 558.412478ms FAIL (2025-07-02 04:36:18 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 558.454677ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 524.507714ms FAIL (2025-07-02 04:35:56 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 524.531763ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 479.845055ms FAIL (2025-07-02 04:35:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 479.873365ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 468.499518ms FAIL (2025-07-02 04:34:25 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 468.532363ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 618.75649ms FAIL (2025-07-02 04:33:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 618.78432ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 458.214295ms FAIL (2025-07-02 04:32:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 458.23905ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 447.529334ms FAIL (2025-07-02 04:31:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 447.55554ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 476.043326ms FAIL (2025-07-02 04:31:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 476.07758ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 475.936133ms FAIL (2025-07-02 04:29:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 475.968405ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 458.795525ms FAIL (2025-07-02 04:29:11 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 458.829253ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 566.40186ms FAIL (2025-07-02 04:27:58 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 566.427372ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 502.83101ms FAIL (2025-07-02 04:27:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 502.859334ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 447.338286ms FAIL (2025-07-02 04:26:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 447.367934ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 642.303872ms FAIL (2025-07-02 04:26:03 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 642.346396ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 454.858318ms FAIL (2025-07-02 04:25:08 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 454.888977ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 436.950088ms FAIL (2025-07-02 04:24:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 437.041055ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 485.704753ms FAIL (2025-07-02 04:23:51 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 485.748541ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 607.450202ms FAIL (2025-07-02 04:23:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 607.479293ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 463.292742ms FAIL (2025-07-02 04:22:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 463.34951ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 450.199092ms FAIL (2025-07-02 04:21:54 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 450.229483ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 458.673115ms FAIL (2025-07-02 04:20:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 458.701043ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 487.499313ms FAIL (2025-07-02 04:20:37 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 487.531108ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 499.621136ms FAIL (2025-07-02 04:20:12 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 499.654232ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 461.782876ms FAIL (2025-07-02 04:19:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 461.815984ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 496.879442ms FAIL (2025-07-02 04:18:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 496.916968ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 412.234602ms FAIL (2025-07-02 04:18:08 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 412.269803ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 466.822696ms FAIL (2025-07-02 04:17:01 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 466.852932ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 503.012566ms FAIL (2025-07-02 04:16:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 503.041986ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 433.352541ms FAIL (2025-07-02 04:15:47 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 433.382039ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 402.830402ms FAIL (2025-07-02 04:14:49 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 402.859376ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 430.617468ms FAIL (2025-07-02 04:14:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 430.656063ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.047507019s FAIL (2025-07-02 04:13:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.047532741s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 513.933138ms FAIL (2025-07-02 03:36:43 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil> Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil> Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments... Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil> Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments... Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment... Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment. Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece... Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece... Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece... Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece... Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece... Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece... Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece... Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece... Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece... Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece... Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece... Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece... Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece... Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece... Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece... Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece... Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments... Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil> Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil> Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil> Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil> Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil> Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil> Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil> Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil> Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil> Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil> Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil> Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil> Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil> Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces. Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments... Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded Duration: 513.975837ms core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)