Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.00131191s FAIL (2025-07-02 07:03:20 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 30.001337409s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 533.611597ms FAIL (2025-07-02 04:57:46 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 533.640957ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 574.92007ms FAIL (2025-07-02 04:57:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 574.965088ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 438.156521ms FAIL (2025-07-02 04:56:25 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 438.193179ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 471.577828ms FAIL (2025-07-02 04:55:16 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 471.618713ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 444.347085ms FAIL (2025-07-02 04:54:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 444.373246ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 430.047324ms FAIL (2025-07-02 04:53:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 430.100088ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 458.944821ms FAIL (2025-07-02 04:52:56 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 458.97791ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 446.328581ms FAIL (2025-07-02 04:52:21 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 446.378341ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 478.597134ms FAIL (2025-07-02 04:51:21 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 478.639908ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 486.934905ms FAIL (2025-07-02 04:50:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 486.981856ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 1.30000927s FAIL (2025-07-02 04:50:03 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 1.300043846s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 415.111787ms FAIL (2025-07-02 04:49:38 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 415.142103ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 402.221396ms FAIL (2025-07-02 04:49:11 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 402.251754ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 498.81698ms FAIL (2025-07-02 04:48:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 498.892924ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 426.310484ms FAIL (2025-07-02 04:48:13 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 426.339324ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 628.01263ms FAIL (2025-07-02 04:47:08 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 628.036358ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 491.522348ms FAIL (2025-07-02 04:46:13 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 491.550184ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 418.774583ms FAIL (2025-07-02 04:45:22 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 418.807292ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 418.689495ms FAIL (2025-07-02 04:44:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 418.718739ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 423.375864ms FAIL (2025-07-02 04:43:51 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 423.45609ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 488.343852ms FAIL (2025-07-02 04:43:01 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 488.377685ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 449.359875ms FAIL (2025-07-02 04:42:06 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 449.396222ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 584.295781ms FAIL (2025-07-02 04:41:05 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 584.325354ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 3.013682862s FAIL (2025-07-02 04:40:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 3.01373898s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 605.377153ms FAIL (2025-07-02 04:40:16 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 605.400779ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 483.294468ms FAIL (2025-07-02 04:39:16 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 483.322104ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 480.695436ms FAIL (2025-07-02 04:38:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 480.727158ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 448.879458ms FAIL (2025-07-02 04:37:27 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 448.911607ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 558.412478ms FAIL (2025-07-02 04:36:18 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 558.454677ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 524.507714ms FAIL (2025-07-02 04:35:56 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 524.531763ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 479.845055ms FAIL (2025-07-02 04:35:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 479.873365ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 468.499518ms FAIL (2025-07-02 04:34:25 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 468.532363ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 618.75649ms FAIL (2025-07-02 04:33:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 618.78432ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 458.214295ms FAIL (2025-07-02 04:32:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 458.23905ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 447.529334ms FAIL (2025-07-02 04:31:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 447.55554ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 476.043326ms FAIL (2025-07-02 04:31:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 476.07758ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 475.936133ms FAIL (2025-07-02 04:29:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 475.968405ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 458.795525ms FAIL (2025-07-02 04:29:11 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 458.829253ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 566.40186ms FAIL (2025-07-02 04:27:58 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 566.427372ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 502.83101ms FAIL (2025-07-02 04:27:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 502.859334ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 447.338286ms FAIL (2025-07-02 04:26:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 447.367934ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 642.303872ms FAIL (2025-07-02 04:26:03 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 642.346396ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 454.858318ms FAIL (2025-07-02 04:25:08 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 454.888977ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 436.950088ms FAIL (2025-07-02 04:24:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 437.041055ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 485.704753ms FAIL (2025-07-02 04:23:51 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 485.748541ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 607.450202ms FAIL (2025-07-02 04:23:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 607.479293ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 463.292742ms FAIL (2025-07-02 04:22:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 463.34951ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 450.199092ms FAIL (2025-07-02 04:21:54 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 450.229483ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 458.673115ms FAIL (2025-07-02 04:20:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== g segments... Jul 2 04:12:45.030542125 (211h1m11.95683682s) [upload=169061]: Upload finished. err: <nil> Jul 2 04:13:10.965456470 (211h1m37.891751147s) [upload=169066]: Starting upload Jul 2 04:13:10.965519651 (211h1m37.891814337s) [upload=169066]: Done starting upload Jul 2 04:13:10.965594008 (211h1m37.891888698s) [upload=169066]: Uploading segments... Jul 2 04:13:10.965864007 (211h1m37.892158689s) [upload=169066]: Got next segment. Inline: false Jul 2 04:13:10.965931585 (211h1m37.892226274s) [upload=169066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 2 04:13:10.965939958 (211h1m37.892234648s) [upload=169066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 2 04:13:10.997048577 (211h1m37.923343267s) [seg_pos={0 0}, upload=169066]: Begin upload segment... Jul 2 04:13:10.997121527 (211h1m37.923416214s) [seg_pos={0 0}, upload=169066]: Done begin upload segment. Jul 2 04:13:10.997184914 (211h1m37.923479606s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.997810592 (211h1m37.924105286s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998393084 (211h1m37.924687779s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.998908968 (211h1m37.925203667s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999412859 (211h1m37.92570756s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:10.999914477 (211h1m37.926209177s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000290603 (211h1m37.926585304s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.000655808 (211h1m37.926950498s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001021435 (211h1m37.927316125s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001396683 (211h1m37.927691381s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.001768813 (211h1m37.928063509s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.002120882 (211h1m37.928415576s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.004711439 (211h1m37.931006136s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005173913 (211h1m37.931468603s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005511785 (211h1m37.931806486s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.005894444 (211h1m37.932189141s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Uploading piece... Jul 2 04:13:11.097809413 (211h1m38.024104092s) [upload=169066]: Next returned nil segment Jul 2 04:13:11.097838146 (211h1m38.024132833s) [upload=169066]: Waiting for error group managing segments... Jul 2 04:13:11.105922992 (211h1m38.03221767s) [noise=true , address=104.36.82.3:4767 , piece_id=NLIL5R65MFAYO5CUNSLG7IMNPXWXOUBT36TRUIHUXV36DUS4MXFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108705010 (211h1m38.034999699s) [noise=true , address=104.36.82.13:4187 , piece_id=72QGCOWMI7D2L4LVOWM3RORSII53M4X7MGQSWBFPGYXLLCACJVKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.108858761 (211h1m38.035153434s) [noise=true , address=104.36.82.16:4295 , piece_id=L2FOQOMWDGUHPDESXXOLGAV4MY23BWHB5IFNCU7FX3YN4P6DVKCQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109473130 (211h1m38.035767816s) [noise=true , address=104.36.82.10:4079 , piece_id=K4NVPDRNU5PXM2WG4FLKOP3N2BYZFWW5DG4R2MCD6WVC6R3EFXHA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.109628197 (211h1m38.035922885s) [noise=true , address=104.36.82.6:4873 , piece_id=RNXYJG6PDAUBAOHFM5VZQKUGW77NJ5V6EZFSXT2H2UFECWVVLWTA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110457373 (211h1m38.036752046s) [noise=true , address=104.36.82.2:4730 , piece_id=UNSXOZKX7OTQXKSP4XCGTTESGED4KMXI6AUJBRLWRRKSN5SCXUKA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.110564351 (211h1m38.036859028s) [noise=true , address=104.36.82.14:4222 , piece_id=TSOYJJTRJZLQU67W4QQC2JSXSC4OV6COLNC77F3ESJ3UFOYQXKNQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:11.477284670 (211h1m38.403579362s) [noise=true , address=104.36.82.5:4840 , piece_id=R525Z3BMZAVELJB44UFA7PAFBV7EXIIWP552ZDK4BEFYM6KBAXJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:30.909223722 (211h1m57.835518408s) [noise=true , address=77.90.31.10:4519 , piece_id=3YPNGRWQECLKPYC3BT6EJ5VZDPR2X74MR7JUZLQL7KLTZL2WDDMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:35.714396607 (211h2m2.640691285s) [noise=true , address=45.91.116.5:4299 , piece_id=OSWUV7SJB3OYM4EF5ZSJLQKDJYDONH773GDL7IKPKE4DSYHDC33Q, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:37.041001580 (211h2m3.967296254s) [noise=true , address=45.91.116.7:4011 , piece_id=AHVLPYLCKQLXIZQVMZUVX7N7BR7VHNQMD5REMZWP7XG3JBVWD5GQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: <nil> Jul 2 04:13:40.934871424 (211h2m7.861166118s) [noise=true , address=45.91.116.11:4119 , piece_id=OVPZUHLVUHQJOAOD6Y2WN5TXY25Y3GWMO4VI4XPPDHHIZNH5DUGA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:59848->45.91.116.11:4119: use of closed network connection Jul 2 04:13:40.934974084 (211h2m7.861268762s) [noise=true , address=77.90.31.16:4734 , piece_id=B7D3Q3X3W2ZM2FHAKPN5XPI4YSOSPDE2IRNGDS2XGA4MKSGXIKKQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:37322->77.90.31.16:4734: use of closed network connection Jul 2 04:13:40.935007706 (211h2m7.861302393s) [noise=true , address=45.91.116.5:4298 , piece_id=TBCQ6CV424U5P4YIS3BUSEYYOQBIZ6JVUUDVOT7XJIY5JFWGTSFQ, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50030->45.91.116.5:4298: use of closed network connection Jul 2 04:13:40.935083810 (211h2m7.861378487s) [noise=true , address=77.90.31.13:4627 , piece_id=6NFSWOQIZUNTA54T4Q6NU2P3OV3QVAYN323YU2GQ4PBGCVXDBOJA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 159.223.143.39:59326->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935152602 (211h2m7.861447292s) [noise=true , address=45.91.116.11:4118 , piece_id=WAXCI4KPRSZY4ETCCUTG3NOLBS2MIHZXU6VMRFBHWT4JE7R5DKMA, seg_pos={0 0}, upload=169066]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 159.223.143.39:50076->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 2 04:13:40.935211981 (211h2m7.861506668s) [seg_pos={0 0}, upload=169066]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 2 04:13:40.980211294 (211h2m7.906505972s) [upload=169066]: Done uploading segments... Jul 2 04:13:40.980234745 (211h2m7.90652942s) [upload=169066]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 458.701043ms core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)