Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 586.542189ms trace: https://jaeger.dev.storj.io/trace/4777dea64937dc1a FAIL (2026-04-06 16:23:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 586.565953ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.XXX........................................................] duration: 550.903228ms trace: https://jaeger.dev.storj.io/trace/28874b4e710a2bcf FAIL (2026-04-06 16:23:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 550.953769ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 1.38002485s trace: https://jaeger.dev.storj.io/trace/449433b6185f502e FAIL (2026-04-06 16:23:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Apr 6 15:40:03.627476193 (748h40m1.224247238s) [upload=838298]: Starting upload Apr 6 15:40:03.627555112 (748h40m1.224326149s) [upload=838298]: Done starting upload Apr 6 15:40:03.627971751 (748h40m1.224742775s) [upload=838298]: Uploading segments... Apr 6 15:40:03.628052062 (748h40m1.224823089s) [upload=838298]: Got next segment. Inline: false Apr 6 15:40:03.628101312 (748h40m1.224872344s) [upload=838298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.628108719 (748h40m1.224879786s) [upload=838298]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.657925511 (748h40m1.254696552s) [seg_pos={0 0}, upload=838298]: Begin upload segment... Apr 6 15:40:03.658006444 (748h40m1.254777507s) [seg_pos={0 0}, upload=838298]: Done begin upload segment. Apr 6 15:40:03.658049451 (748h40m1.254820484s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658127629 (748h40m1.254898659s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658849378 (748h40m1.255620415s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.659549528 (748h40m1.256320563s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660187950 (748h40m1.256958976s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660848819 (748h40m1.257619849s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.661744944 (748h40m1.25851598s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.662381990 (748h40m1.259153021s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.663016005 (748h40m1.25978704s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.665366739 (748h40m1.262137771s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.666327038 (748h40m1.263098052s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.667370344 (748h40m1.264141368s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.668228606 (748h40m1.264999623s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.670639310 (748h40m1.267410325s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.671832750 (748h40m1.268603781s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.672735177 (748h40m1.269506187s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.673761153 (748h40m1.270532166s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.674068775 (748h40m1.27083979s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.768526476 (748h40m1.365297491s) [upload=838298]: Next returned nil segment Apr 6 15:40:03.768546305 (748h40m1.365317356s) [upload=838298]: Waiting for error group managing segments... Apr 6 15:40:04.090183416 (748h40m1.686954485s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111363426 (748h40m1.708134444s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111628225 (748h40m1.708399237s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.123286299 (748h40m1.720057314s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.305166549 (748h40m1.901937573s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.309277855 (748h40m1.906048873s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.310384222 (748h40m1.90715524s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.311558249 (748h40m1.908329268s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.318896832 (748h40m1.915667854s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.319687548 (748h40m1.916458575s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.320573940 (748h40m1.917344962s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.694686967 (748h40m2.291457994s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.695909952 (748h40m2.292680979s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:41:03.579925561 (748h41m1.176696593s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: rpc: dial tcp 45.91.116.13:4181: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580475482 (748h41m1.177246511s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12UvDrJhqPFda5Vu8NEGBVBYXHoXLFWnKHNJqrLLGgfVi14sZsJ, address:45.91.116.5:4331): protocol: write tcp 159.223.143.39:55764->45.91.116.5:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580772045 (748h41m1.177543071s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 159.223.143.39:33530->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581261292 (748h41m1.178032318s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12CuCqHJYbanbuaEMHvtcuQXhvcdahD8eyT2fKDhqtxdSZ6ES4f, address:45.91.116.6:4000): protocol: write tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; close tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581606038 (748h41m1.178377062s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1aarPYLumhCAtisqFVYu9iFnZqBHRFMFN2xakzn48kapVp2ae5, address:45.91.116.14:4197): protocol: context deadline exceeded; context deadline exceeded Apr 6 15:41:03.581732624 (748h41m1.178503641s) [seg_pos={0 0}, upload=838298]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.605380574 (748h41m1.202151599s) [upload=838298]: Done uploading segments... Apr 6 15:41:03.605444334 (748h41m1.20221536s) [upload=838298]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.380125483s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 589.756253ms trace: https://jaeger.dev.storj.io/trace/4777dea64937dc1a FAIL (2026-04-06 16:23:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 589.782665ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [..XXX.......................................................] duration: 717.134347ms trace: https://jaeger.dev.storj.io/trace/28874b4e710a2bcf FAIL (2026-04-06 16:22:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 717.224211ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 1.326261805s trace: https://jaeger.dev.storj.io/trace/449433b6185f502e FAIL (2026-04-06 16:22:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Apr 6 15:40:03.627476193 (748h40m1.224247238s) [upload=838298]: Starting upload Apr 6 15:40:03.627555112 (748h40m1.224326149s) [upload=838298]: Done starting upload Apr 6 15:40:03.627971751 (748h40m1.224742775s) [upload=838298]: Uploading segments... Apr 6 15:40:03.628052062 (748h40m1.224823089s) [upload=838298]: Got next segment. Inline: false Apr 6 15:40:03.628101312 (748h40m1.224872344s) [upload=838298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.628108719 (748h40m1.224879786s) [upload=838298]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.657925511 (748h40m1.254696552s) [seg_pos={0 0}, upload=838298]: Begin upload segment... Apr 6 15:40:03.658006444 (748h40m1.254777507s) [seg_pos={0 0}, upload=838298]: Done begin upload segment. Apr 6 15:40:03.658049451 (748h40m1.254820484s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658127629 (748h40m1.254898659s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658849378 (748h40m1.255620415s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.659549528 (748h40m1.256320563s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660187950 (748h40m1.256958976s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660848819 (748h40m1.257619849s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.661744944 (748h40m1.25851598s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.662381990 (748h40m1.259153021s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.663016005 (748h40m1.25978704s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.665366739 (748h40m1.262137771s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.666327038 (748h40m1.263098052s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.667370344 (748h40m1.264141368s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.668228606 (748h40m1.264999623s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.670639310 (748h40m1.267410325s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.671832750 (748h40m1.268603781s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.672735177 (748h40m1.269506187s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.673761153 (748h40m1.270532166s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.674068775 (748h40m1.27083979s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.768526476 (748h40m1.365297491s) [upload=838298]: Next returned nil segment Apr 6 15:40:03.768546305 (748h40m1.365317356s) [upload=838298]: Waiting for error group managing segments... Apr 6 15:40:04.090183416 (748h40m1.686954485s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111363426 (748h40m1.708134444s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111628225 (748h40m1.708399237s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.123286299 (748h40m1.720057314s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.305166549 (748h40m1.901937573s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.309277855 (748h40m1.906048873s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.310384222 (748h40m1.90715524s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.311558249 (748h40m1.908329268s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.318896832 (748h40m1.915667854s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.319687548 (748h40m1.916458575s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.320573940 (748h40m1.917344962s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.694686967 (748h40m2.291457994s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.695909952 (748h40m2.292680979s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:41:03.579925561 (748h41m1.176696593s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: rpc: dial tcp 45.91.116.13:4181: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580475482 (748h41m1.177246511s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12UvDrJhqPFda5Vu8NEGBVBYXHoXLFWnKHNJqrLLGgfVi14sZsJ, address:45.91.116.5:4331): protocol: write tcp 159.223.143.39:55764->45.91.116.5:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580772045 (748h41m1.177543071s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 159.223.143.39:33530->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581261292 (748h41m1.178032318s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12CuCqHJYbanbuaEMHvtcuQXhvcdahD8eyT2fKDhqtxdSZ6ES4f, address:45.91.116.6:4000): protocol: write tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; close tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581606038 (748h41m1.178377062s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1aarPYLumhCAtisqFVYu9iFnZqBHRFMFN2xakzn48kapVp2ae5, address:45.91.116.14:4197): protocol: context deadline exceeded; context deadline exceeded Apr 6 15:41:03.581732624 (748h41m1.178503641s) [seg_pos={0 0}, upload=838298]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.605380574 (748h41m1.202151599s) [upload=838298]: Done uploading segments... Apr 6 15:41:03.605444334 (748h41m1.20221536s) [upload=838298]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.326293497s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [...XXX......................................................] duration: 663.152693ms trace: https://jaeger.dev.storj.io/trace/28874b4e710a2bcf FAIL (2026-04-06 16:22:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 663.184083ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 621.997902ms trace: https://jaeger.dev.storj.io/trace/4777dea64937dc1a FAIL (2026-04-06 16:22:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 622.030599ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 1.012805217s trace: https://jaeger.dev.storj.io/trace/449433b6185f502e FAIL (2026-04-06 16:21:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Apr 6 15:40:03.627476193 (748h40m1.224247238s) [upload=838298]: Starting upload Apr 6 15:40:03.627555112 (748h40m1.224326149s) [upload=838298]: Done starting upload Apr 6 15:40:03.627971751 (748h40m1.224742775s) [upload=838298]: Uploading segments... Apr 6 15:40:03.628052062 (748h40m1.224823089s) [upload=838298]: Got next segment. Inline: false Apr 6 15:40:03.628101312 (748h40m1.224872344s) [upload=838298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.628108719 (748h40m1.224879786s) [upload=838298]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.657925511 (748h40m1.254696552s) [seg_pos={0 0}, upload=838298]: Begin upload segment... Apr 6 15:40:03.658006444 (748h40m1.254777507s) [seg_pos={0 0}, upload=838298]: Done begin upload segment. Apr 6 15:40:03.658049451 (748h40m1.254820484s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658127629 (748h40m1.254898659s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658849378 (748h40m1.255620415s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.659549528 (748h40m1.256320563s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660187950 (748h40m1.256958976s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660848819 (748h40m1.257619849s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.661744944 (748h40m1.25851598s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.662381990 (748h40m1.259153021s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.663016005 (748h40m1.25978704s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.665366739 (748h40m1.262137771s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.666327038 (748h40m1.263098052s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.667370344 (748h40m1.264141368s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.668228606 (748h40m1.264999623s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.670639310 (748h40m1.267410325s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.671832750 (748h40m1.268603781s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.672735177 (748h40m1.269506187s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.673761153 (748h40m1.270532166s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.674068775 (748h40m1.27083979s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.768526476 (748h40m1.365297491s) [upload=838298]: Next returned nil segment Apr 6 15:40:03.768546305 (748h40m1.365317356s) [upload=838298]: Waiting for error group managing segments... Apr 6 15:40:04.090183416 (748h40m1.686954485s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111363426 (748h40m1.708134444s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111628225 (748h40m1.708399237s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.123286299 (748h40m1.720057314s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.305166549 (748h40m1.901937573s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.309277855 (748h40m1.906048873s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.310384222 (748h40m1.90715524s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.311558249 (748h40m1.908329268s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.318896832 (748h40m1.915667854s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.319687548 (748h40m1.916458575s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.320573940 (748h40m1.917344962s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.694686967 (748h40m2.291457994s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.695909952 (748h40m2.292680979s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:41:03.579925561 (748h41m1.176696593s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: rpc: dial tcp 45.91.116.13:4181: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580475482 (748h41m1.177246511s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12UvDrJhqPFda5Vu8NEGBVBYXHoXLFWnKHNJqrLLGgfVi14sZsJ, address:45.91.116.5:4331): protocol: write tcp 159.223.143.39:55764->45.91.116.5:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580772045 (748h41m1.177543071s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 159.223.143.39:33530->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581261292 (748h41m1.178032318s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12CuCqHJYbanbuaEMHvtcuQXhvcdahD8eyT2fKDhqtxdSZ6ES4f, address:45.91.116.6:4000): protocol: write tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; close tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581606038 (748h41m1.178377062s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1aarPYLumhCAtisqFVYu9iFnZqBHRFMFN2xakzn48kapVp2ae5, address:45.91.116.14:4197): protocol: context deadline exceeded; context deadline exceeded Apr 6 15:41:03.581732624 (748h41m1.178503641s) [seg_pos={0 0}, upload=838298]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.605380574 (748h41m1.202151599s) [upload=838298]: Done uploading segments... Apr 6 15:41:03.605444334 (748h41m1.20221536s) [upload=838298]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.012901107s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [....XXX.....................................................] duration: 540.760352ms trace: https://jaeger.dev.storj.io/trace/28874b4e710a2bcf FAIL (2026-04-06 16:20:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 540.796542ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 654.875735ms trace: https://jaeger.dev.storj.io/trace/4777dea64937dc1a FAIL (2026-04-06 16:20:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 654.912082ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 1.256975505s trace: https://jaeger.dev.storj.io/trace/449433b6185f502e FAIL (2026-04-06 16:20:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Apr 6 15:40:03.627476193 (748h40m1.224247238s) [upload=838298]: Starting upload Apr 6 15:40:03.627555112 (748h40m1.224326149s) [upload=838298]: Done starting upload Apr 6 15:40:03.627971751 (748h40m1.224742775s) [upload=838298]: Uploading segments... Apr 6 15:40:03.628052062 (748h40m1.224823089s) [upload=838298]: Got next segment. Inline: false Apr 6 15:40:03.628101312 (748h40m1.224872344s) [upload=838298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.628108719 (748h40m1.224879786s) [upload=838298]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.657925511 (748h40m1.254696552s) [seg_pos={0 0}, upload=838298]: Begin upload segment... Apr 6 15:40:03.658006444 (748h40m1.254777507s) [seg_pos={0 0}, upload=838298]: Done begin upload segment. Apr 6 15:40:03.658049451 (748h40m1.254820484s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658127629 (748h40m1.254898659s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658849378 (748h40m1.255620415s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.659549528 (748h40m1.256320563s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660187950 (748h40m1.256958976s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660848819 (748h40m1.257619849s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.661744944 (748h40m1.25851598s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.662381990 (748h40m1.259153021s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.663016005 (748h40m1.25978704s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.665366739 (748h40m1.262137771s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.666327038 (748h40m1.263098052s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.667370344 (748h40m1.264141368s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.668228606 (748h40m1.264999623s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.670639310 (748h40m1.267410325s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.671832750 (748h40m1.268603781s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.672735177 (748h40m1.269506187s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.673761153 (748h40m1.270532166s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.674068775 (748h40m1.27083979s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.768526476 (748h40m1.365297491s) [upload=838298]: Next returned nil segment Apr 6 15:40:03.768546305 (748h40m1.365317356s) [upload=838298]: Waiting for error group managing segments... Apr 6 15:40:04.090183416 (748h40m1.686954485s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111363426 (748h40m1.708134444s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111628225 (748h40m1.708399237s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.123286299 (748h40m1.720057314s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.305166549 (748h40m1.901937573s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.309277855 (748h40m1.906048873s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.310384222 (748h40m1.90715524s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.311558249 (748h40m1.908329268s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.318896832 (748h40m1.915667854s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.319687548 (748h40m1.916458575s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.320573940 (748h40m1.917344962s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.694686967 (748h40m2.291457994s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.695909952 (748h40m2.292680979s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:41:03.579925561 (748h41m1.176696593s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: rpc: dial tcp 45.91.116.13:4181: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580475482 (748h41m1.177246511s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12UvDrJhqPFda5Vu8NEGBVBYXHoXLFWnKHNJqrLLGgfVi14sZsJ, address:45.91.116.5:4331): protocol: write tcp 159.223.143.39:55764->45.91.116.5:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580772045 (748h41m1.177543071s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 159.223.143.39:33530->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581261292 (748h41m1.178032318s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12CuCqHJYbanbuaEMHvtcuQXhvcdahD8eyT2fKDhqtxdSZ6ES4f, address:45.91.116.6:4000): protocol: write tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; close tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581606038 (748h41m1.178377062s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1aarPYLumhCAtisqFVYu9iFnZqBHRFMFN2xakzn48kapVp2ae5, address:45.91.116.14:4197): protocol: context deadline exceeded; context deadline exceeded Apr 6 15:41:03.581732624 (748h41m1.178503641s) [seg_pos={0 0}, upload=838298]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.605380574 (748h41m1.202151599s) [upload=838298]: Done uploading segments... Apr 6 15:41:03.605444334 (748h41m1.20221536s) [upload=838298]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.257059918s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 536.838995ms trace: https://jaeger.dev.storj.io/trace/4777dea64937dc1a FAIL (2026-04-06 16:20:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 536.869495ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.....XXX....................................................] duration: 803.706965ms trace: https://jaeger.dev.storj.io/trace/28874b4e710a2bcf FAIL (2026-04-06 16:20:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 803.735517ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 1.255289995s trace: https://jaeger.dev.storj.io/trace/449433b6185f502e FAIL (2026-04-06 16:19:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Apr 6 15:40:03.627476193 (748h40m1.224247238s) [upload=838298]: Starting upload Apr 6 15:40:03.627555112 (748h40m1.224326149s) [upload=838298]: Done starting upload Apr 6 15:40:03.627971751 (748h40m1.224742775s) [upload=838298]: Uploading segments... Apr 6 15:40:03.628052062 (748h40m1.224823089s) [upload=838298]: Got next segment. Inline: false Apr 6 15:40:03.628101312 (748h40m1.224872344s) [upload=838298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.628108719 (748h40m1.224879786s) [upload=838298]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.657925511 (748h40m1.254696552s) [seg_pos={0 0}, upload=838298]: Begin upload segment... Apr 6 15:40:03.658006444 (748h40m1.254777507s) [seg_pos={0 0}, upload=838298]: Done begin upload segment. Apr 6 15:40:03.658049451 (748h40m1.254820484s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658127629 (748h40m1.254898659s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658849378 (748h40m1.255620415s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.659549528 (748h40m1.256320563s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660187950 (748h40m1.256958976s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660848819 (748h40m1.257619849s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.661744944 (748h40m1.25851598s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.662381990 (748h40m1.259153021s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.663016005 (748h40m1.25978704s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.665366739 (748h40m1.262137771s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.666327038 (748h40m1.263098052s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.667370344 (748h40m1.264141368s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.668228606 (748h40m1.264999623s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.670639310 (748h40m1.267410325s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.671832750 (748h40m1.268603781s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.672735177 (748h40m1.269506187s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.673761153 (748h40m1.270532166s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.674068775 (748h40m1.27083979s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.768526476 (748h40m1.365297491s) [upload=838298]: Next returned nil segment Apr 6 15:40:03.768546305 (748h40m1.365317356s) [upload=838298]: Waiting for error group managing segments... Apr 6 15:40:04.090183416 (748h40m1.686954485s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111363426 (748h40m1.708134444s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111628225 (748h40m1.708399237s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.123286299 (748h40m1.720057314s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.305166549 (748h40m1.901937573s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.309277855 (748h40m1.906048873s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.310384222 (748h40m1.90715524s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.311558249 (748h40m1.908329268s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.318896832 (748h40m1.915667854s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.319687548 (748h40m1.916458575s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.320573940 (748h40m1.917344962s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.694686967 (748h40m2.291457994s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.695909952 (748h40m2.292680979s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:41:03.579925561 (748h41m1.176696593s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: rpc: dial tcp 45.91.116.13:4181: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580475482 (748h41m1.177246511s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12UvDrJhqPFda5Vu8NEGBVBYXHoXLFWnKHNJqrLLGgfVi14sZsJ, address:45.91.116.5:4331): protocol: write tcp 159.223.143.39:55764->45.91.116.5:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580772045 (748h41m1.177543071s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 159.223.143.39:33530->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581261292 (748h41m1.178032318s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12CuCqHJYbanbuaEMHvtcuQXhvcdahD8eyT2fKDhqtxdSZ6ES4f, address:45.91.116.6:4000): protocol: write tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; close tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581606038 (748h41m1.178377062s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1aarPYLumhCAtisqFVYu9iFnZqBHRFMFN2xakzn48kapVp2ae5, address:45.91.116.14:4197): protocol: context deadline exceeded; context deadline exceeded Apr 6 15:41:03.581732624 (748h41m1.178503641s) [seg_pos={0 0}, upload=838298]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.605380574 (748h41m1.202151599s) [upload=838298]: Done uploading segments... Apr 6 15:41:03.605444334 (748h41m1.20221536s) [upload=838298]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.255338545s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 786.01626ms trace: https://jaeger.dev.storj.io/trace/4777dea64937dc1a FAIL (2026-04-06 16:19:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 786.145071ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [......XXX...................................................] duration: 582.620015ms trace: https://jaeger.dev.storj.io/trace/28874b4e710a2bcf FAIL (2026-04-06 16:19:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 582.668156ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 1.322249854s trace: https://jaeger.dev.storj.io/trace/449433b6185f502e FAIL (2026-04-06 16:19:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Apr 6 15:40:03.627476193 (748h40m1.224247238s) [upload=838298]: Starting upload Apr 6 15:40:03.627555112 (748h40m1.224326149s) [upload=838298]: Done starting upload Apr 6 15:40:03.627971751 (748h40m1.224742775s) [upload=838298]: Uploading segments... Apr 6 15:40:03.628052062 (748h40m1.224823089s) [upload=838298]: Got next segment. Inline: false Apr 6 15:40:03.628101312 (748h40m1.224872344s) [upload=838298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.628108719 (748h40m1.224879786s) [upload=838298]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.657925511 (748h40m1.254696552s) [seg_pos={0 0}, upload=838298]: Begin upload segment... Apr 6 15:40:03.658006444 (748h40m1.254777507s) [seg_pos={0 0}, upload=838298]: Done begin upload segment. Apr 6 15:40:03.658049451 (748h40m1.254820484s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658127629 (748h40m1.254898659s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658849378 (748h40m1.255620415s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.659549528 (748h40m1.256320563s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660187950 (748h40m1.256958976s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660848819 (748h40m1.257619849s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.661744944 (748h40m1.25851598s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.662381990 (748h40m1.259153021s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.663016005 (748h40m1.25978704s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.665366739 (748h40m1.262137771s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.666327038 (748h40m1.263098052s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.667370344 (748h40m1.264141368s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.668228606 (748h40m1.264999623s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.670639310 (748h40m1.267410325s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.671832750 (748h40m1.268603781s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.672735177 (748h40m1.269506187s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.673761153 (748h40m1.270532166s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.674068775 (748h40m1.27083979s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.768526476 (748h40m1.365297491s) [upload=838298]: Next returned nil segment Apr 6 15:40:03.768546305 (748h40m1.365317356s) [upload=838298]: Waiting for error group managing segments... Apr 6 15:40:04.090183416 (748h40m1.686954485s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111363426 (748h40m1.708134444s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111628225 (748h40m1.708399237s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.123286299 (748h40m1.720057314s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.305166549 (748h40m1.901937573s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.309277855 (748h40m1.906048873s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.310384222 (748h40m1.90715524s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.311558249 (748h40m1.908329268s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.318896832 (748h40m1.915667854s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.319687548 (748h40m1.916458575s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.320573940 (748h40m1.917344962s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.694686967 (748h40m2.291457994s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.695909952 (748h40m2.292680979s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:41:03.579925561 (748h41m1.176696593s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: rpc: dial tcp 45.91.116.13:4181: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580475482 (748h41m1.177246511s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12UvDrJhqPFda5Vu8NEGBVBYXHoXLFWnKHNJqrLLGgfVi14sZsJ, address:45.91.116.5:4331): protocol: write tcp 159.223.143.39:55764->45.91.116.5:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580772045 (748h41m1.177543071s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 159.223.143.39:33530->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581261292 (748h41m1.178032318s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12CuCqHJYbanbuaEMHvtcuQXhvcdahD8eyT2fKDhqtxdSZ6ES4f, address:45.91.116.6:4000): protocol: write tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; close tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581606038 (748h41m1.178377062s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1aarPYLumhCAtisqFVYu9iFnZqBHRFMFN2xakzn48kapVp2ae5, address:45.91.116.14:4197): protocol: context deadline exceeded; context deadline exceeded Apr 6 15:41:03.581732624 (748h41m1.178503641s) [seg_pos={0 0}, upload=838298]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.605380574 (748h41m1.202151599s) [upload=838298]: Done uploading segments... Apr 6 15:41:03.605444334 (748h41m1.20221536s) [upload=838298]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.32231536s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.......XXX..................................................] duration: 750.069546ms trace: https://jaeger.dev.storj.io/trace/28874b4e710a2bcf FAIL (2026-04-06 16:18:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 750.109345ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 1.388146979s trace: https://jaeger.dev.storj.io/trace/449433b6185f502e FAIL (2026-04-06 16:18:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Apr 6 15:40:03.627476193 (748h40m1.224247238s) [upload=838298]: Starting upload Apr 6 15:40:03.627555112 (748h40m1.224326149s) [upload=838298]: Done starting upload Apr 6 15:40:03.627971751 (748h40m1.224742775s) [upload=838298]: Uploading segments... Apr 6 15:40:03.628052062 (748h40m1.224823089s) [upload=838298]: Got next segment. Inline: false Apr 6 15:40:03.628101312 (748h40m1.224872344s) [upload=838298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.628108719 (748h40m1.224879786s) [upload=838298]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.657925511 (748h40m1.254696552s) [seg_pos={0 0}, upload=838298]: Begin upload segment... Apr 6 15:40:03.658006444 (748h40m1.254777507s) [seg_pos={0 0}, upload=838298]: Done begin upload segment. Apr 6 15:40:03.658049451 (748h40m1.254820484s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658127629 (748h40m1.254898659s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658849378 (748h40m1.255620415s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.659549528 (748h40m1.256320563s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660187950 (748h40m1.256958976s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660848819 (748h40m1.257619849s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.661744944 (748h40m1.25851598s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.662381990 (748h40m1.259153021s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.663016005 (748h40m1.25978704s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.665366739 (748h40m1.262137771s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.666327038 (748h40m1.263098052s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.667370344 (748h40m1.264141368s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.668228606 (748h40m1.264999623s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.670639310 (748h40m1.267410325s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.671832750 (748h40m1.268603781s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.672735177 (748h40m1.269506187s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.673761153 (748h40m1.270532166s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.674068775 (748h40m1.27083979s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.768526476 (748h40m1.365297491s) [upload=838298]: Next returned nil segment Apr 6 15:40:03.768546305 (748h40m1.365317356s) [upload=838298]: Waiting for error group managing segments... Apr 6 15:40:04.090183416 (748h40m1.686954485s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111363426 (748h40m1.708134444s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111628225 (748h40m1.708399237s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.123286299 (748h40m1.720057314s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.305166549 (748h40m1.901937573s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.309277855 (748h40m1.906048873s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.310384222 (748h40m1.90715524s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.311558249 (748h40m1.908329268s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.318896832 (748h40m1.915667854s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.319687548 (748h40m1.916458575s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.320573940 (748h40m1.917344962s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.694686967 (748h40m2.291457994s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.695909952 (748h40m2.292680979s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:41:03.579925561 (748h41m1.176696593s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: rpc: dial tcp 45.91.116.13:4181: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580475482 (748h41m1.177246511s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12UvDrJhqPFda5Vu8NEGBVBYXHoXLFWnKHNJqrLLGgfVi14sZsJ, address:45.91.116.5:4331): protocol: write tcp 159.223.143.39:55764->45.91.116.5:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580772045 (748h41m1.177543071s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 159.223.143.39:33530->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581261292 (748h41m1.178032318s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12CuCqHJYbanbuaEMHvtcuQXhvcdahD8eyT2fKDhqtxdSZ6ES4f, address:45.91.116.6:4000): protocol: write tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; close tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581606038 (748h41m1.178377062s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1aarPYLumhCAtisqFVYu9iFnZqBHRFMFN2xakzn48kapVp2ae5, address:45.91.116.14:4197): protocol: context deadline exceeded; context deadline exceeded Apr 6 15:41:03.581732624 (748h41m1.178503641s) [seg_pos={0 0}, upload=838298]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.605380574 (748h41m1.202151599s) [upload=838298]: Done uploading segments... Apr 6 15:41:03.605444334 (748h41m1.20221536s) [upload=838298]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.38817638s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [........XXX.................................................] duration: 610.296053ms trace: https://jaeger.dev.storj.io/trace/28874b4e710a2bcf FAIL (2026-04-06 16:18:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 610.335318ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 867.671423ms trace: https://jaeger.dev.storj.io/trace/4777dea64937dc1a FAIL (2026-04-06 16:18:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 867.735821ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 1.209517784s trace: https://jaeger.dev.storj.io/trace/449433b6185f502e FAIL (2026-04-06 16:18:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Apr 6 15:40:03.627476193 (748h40m1.224247238s) [upload=838298]: Starting upload Apr 6 15:40:03.627555112 (748h40m1.224326149s) [upload=838298]: Done starting upload Apr 6 15:40:03.627971751 (748h40m1.224742775s) [upload=838298]: Uploading segments... Apr 6 15:40:03.628052062 (748h40m1.224823089s) [upload=838298]: Got next segment. Inline: false Apr 6 15:40:03.628101312 (748h40m1.224872344s) [upload=838298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.628108719 (748h40m1.224879786s) [upload=838298]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.657925511 (748h40m1.254696552s) [seg_pos={0 0}, upload=838298]: Begin upload segment... Apr 6 15:40:03.658006444 (748h40m1.254777507s) [seg_pos={0 0}, upload=838298]: Done begin upload segment. Apr 6 15:40:03.658049451 (748h40m1.254820484s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658127629 (748h40m1.254898659s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658849378 (748h40m1.255620415s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.659549528 (748h40m1.256320563s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660187950 (748h40m1.256958976s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660848819 (748h40m1.257619849s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.661744944 (748h40m1.25851598s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.662381990 (748h40m1.259153021s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.663016005 (748h40m1.25978704s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.665366739 (748h40m1.262137771s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.666327038 (748h40m1.263098052s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.667370344 (748h40m1.264141368s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.668228606 (748h40m1.264999623s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.670639310 (748h40m1.267410325s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.671832750 (748h40m1.268603781s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.672735177 (748h40m1.269506187s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.673761153 (748h40m1.270532166s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.674068775 (748h40m1.27083979s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.768526476 (748h40m1.365297491s) [upload=838298]: Next returned nil segment Apr 6 15:40:03.768546305 (748h40m1.365317356s) [upload=838298]: Waiting for error group managing segments... Apr 6 15:40:04.090183416 (748h40m1.686954485s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111363426 (748h40m1.708134444s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111628225 (748h40m1.708399237s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.123286299 (748h40m1.720057314s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.305166549 (748h40m1.901937573s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.309277855 (748h40m1.906048873s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.310384222 (748h40m1.90715524s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.311558249 (748h40m1.908329268s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.318896832 (748h40m1.915667854s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.319687548 (748h40m1.916458575s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.320573940 (748h40m1.917344962s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.694686967 (748h40m2.291457994s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.695909952 (748h40m2.292680979s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:41:03.579925561 (748h41m1.176696593s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: rpc: dial tcp 45.91.116.13:4181: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580475482 (748h41m1.177246511s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12UvDrJhqPFda5Vu8NEGBVBYXHoXLFWnKHNJqrLLGgfVi14sZsJ, address:45.91.116.5:4331): protocol: write tcp 159.223.143.39:55764->45.91.116.5:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580772045 (748h41m1.177543071s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 159.223.143.39:33530->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581261292 (748h41m1.178032318s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12CuCqHJYbanbuaEMHvtcuQXhvcdahD8eyT2fKDhqtxdSZ6ES4f, address:45.91.116.6:4000): protocol: write tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; close tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581606038 (748h41m1.178377062s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1aarPYLumhCAtisqFVYu9iFnZqBHRFMFN2xakzn48kapVp2ae5, address:45.91.116.14:4197): protocol: context deadline exceeded; context deadline exceeded Apr 6 15:41:03.581732624 (748h41m1.178503641s) [seg_pos={0 0}, upload=838298]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.605380574 (748h41m1.202151599s) [upload=838298]: Done uploading segments... Apr 6 15:41:03.605444334 (748h41m1.20221536s) [upload=838298]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.209589308s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 1.568699934s trace: https://jaeger.dev.storj.io/trace/449433b6185f502e FAIL (2026-04-06 16:17:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Apr 6 15:40:03.627476193 (748h40m1.224247238s) [upload=838298]: Starting upload Apr 6 15:40:03.627555112 (748h40m1.224326149s) [upload=838298]: Done starting upload Apr 6 15:40:03.627971751 (748h40m1.224742775s) [upload=838298]: Uploading segments... Apr 6 15:40:03.628052062 (748h40m1.224823089s) [upload=838298]: Got next segment. Inline: false Apr 6 15:40:03.628101312 (748h40m1.224872344s) [upload=838298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.628108719 (748h40m1.224879786s) [upload=838298]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.657925511 (748h40m1.254696552s) [seg_pos={0 0}, upload=838298]: Begin upload segment... Apr 6 15:40:03.658006444 (748h40m1.254777507s) [seg_pos={0 0}, upload=838298]: Done begin upload segment. Apr 6 15:40:03.658049451 (748h40m1.254820484s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658127629 (748h40m1.254898659s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658849378 (748h40m1.255620415s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.659549528 (748h40m1.256320563s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660187950 (748h40m1.256958976s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660848819 (748h40m1.257619849s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.661744944 (748h40m1.25851598s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.662381990 (748h40m1.259153021s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.663016005 (748h40m1.25978704s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.665366739 (748h40m1.262137771s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.666327038 (748h40m1.263098052s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.667370344 (748h40m1.264141368s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.668228606 (748h40m1.264999623s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.670639310 (748h40m1.267410325s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.671832750 (748h40m1.268603781s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.672735177 (748h40m1.269506187s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.673761153 (748h40m1.270532166s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.674068775 (748h40m1.27083979s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.768526476 (748h40m1.365297491s) [upload=838298]: Next returned nil segment Apr 6 15:40:03.768546305 (748h40m1.365317356s) [upload=838298]: Waiting for error group managing segments... Apr 6 15:40:04.090183416 (748h40m1.686954485s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111363426 (748h40m1.708134444s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111628225 (748h40m1.708399237s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.123286299 (748h40m1.720057314s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.305166549 (748h40m1.901937573s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.309277855 (748h40m1.906048873s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.310384222 (748h40m1.90715524s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.311558249 (748h40m1.908329268s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.318896832 (748h40m1.915667854s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.319687548 (748h40m1.916458575s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.320573940 (748h40m1.917344962s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.694686967 (748h40m2.291457994s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.695909952 (748h40m2.292680979s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:41:03.579925561 (748h41m1.176696593s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: rpc: dial tcp 45.91.116.13:4181: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580475482 (748h41m1.177246511s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12UvDrJhqPFda5Vu8NEGBVBYXHoXLFWnKHNJqrLLGgfVi14sZsJ, address:45.91.116.5:4331): protocol: write tcp 159.223.143.39:55764->45.91.116.5:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580772045 (748h41m1.177543071s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 159.223.143.39:33530->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581261292 (748h41m1.178032318s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12CuCqHJYbanbuaEMHvtcuQXhvcdahD8eyT2fKDhqtxdSZ6ES4f, address:45.91.116.6:4000): protocol: write tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; close tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581606038 (748h41m1.178377062s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1aarPYLumhCAtisqFVYu9iFnZqBHRFMFN2xakzn48kapVp2ae5, address:45.91.116.14:4197): protocol: context deadline exceeded; context deadline exceeded Apr 6 15:41:03.581732624 (748h41m1.178503641s) [seg_pos={0 0}, upload=838298]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.605380574 (748h41m1.202151599s) [upload=838298]: Done uploading segments... Apr 6 15:41:03.605444334 (748h41m1.20221536s) [upload=838298]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.568724254s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.........XXX................................................] duration: 600.142662ms trace: https://jaeger.dev.storj.io/trace/28874b4e710a2bcf FAIL (2026-04-06 16:17:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 600.195023ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 729.598813ms trace: https://jaeger.dev.storj.io/trace/4777dea64937dc1a FAIL (2026-04-06 16:17:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 729.637658ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 1.275954806s trace: https://jaeger.dev.storj.io/trace/449433b6185f502e FAIL (2026-04-06 16:17:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Apr 6 15:40:03.627476193 (748h40m1.224247238s) [upload=838298]: Starting upload Apr 6 15:40:03.627555112 (748h40m1.224326149s) [upload=838298]: Done starting upload Apr 6 15:40:03.627971751 (748h40m1.224742775s) [upload=838298]: Uploading segments... Apr 6 15:40:03.628052062 (748h40m1.224823089s) [upload=838298]: Got next segment. Inline: false Apr 6 15:40:03.628101312 (748h40m1.224872344s) [upload=838298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.628108719 (748h40m1.224879786s) [upload=838298]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.657925511 (748h40m1.254696552s) [seg_pos={0 0}, upload=838298]: Begin upload segment... Apr 6 15:40:03.658006444 (748h40m1.254777507s) [seg_pos={0 0}, upload=838298]: Done begin upload segment. Apr 6 15:40:03.658049451 (748h40m1.254820484s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658127629 (748h40m1.254898659s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658849378 (748h40m1.255620415s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.659549528 (748h40m1.256320563s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660187950 (748h40m1.256958976s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660848819 (748h40m1.257619849s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.661744944 (748h40m1.25851598s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.662381990 (748h40m1.259153021s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.663016005 (748h40m1.25978704s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.665366739 (748h40m1.262137771s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.666327038 (748h40m1.263098052s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.667370344 (748h40m1.264141368s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.668228606 (748h40m1.264999623s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.670639310 (748h40m1.267410325s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.671832750 (748h40m1.268603781s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.672735177 (748h40m1.269506187s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.673761153 (748h40m1.270532166s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.674068775 (748h40m1.27083979s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.768526476 (748h40m1.365297491s) [upload=838298]: Next returned nil segment Apr 6 15:40:03.768546305 (748h40m1.365317356s) [upload=838298]: Waiting for error group managing segments... Apr 6 15:40:04.090183416 (748h40m1.686954485s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111363426 (748h40m1.708134444s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111628225 (748h40m1.708399237s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.123286299 (748h40m1.720057314s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.305166549 (748h40m1.901937573s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.309277855 (748h40m1.906048873s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.310384222 (748h40m1.90715524s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.311558249 (748h40m1.908329268s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.318896832 (748h40m1.915667854s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.319687548 (748h40m1.916458575s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.320573940 (748h40m1.917344962s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.694686967 (748h40m2.291457994s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.695909952 (748h40m2.292680979s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:41:03.579925561 (748h41m1.176696593s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: rpc: dial tcp 45.91.116.13:4181: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580475482 (748h41m1.177246511s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12UvDrJhqPFda5Vu8NEGBVBYXHoXLFWnKHNJqrLLGgfVi14sZsJ, address:45.91.116.5:4331): protocol: write tcp 159.223.143.39:55764->45.91.116.5:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580772045 (748h41m1.177543071s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 159.223.143.39:33530->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581261292 (748h41m1.178032318s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12CuCqHJYbanbuaEMHvtcuQXhvcdahD8eyT2fKDhqtxdSZ6ES4f, address:45.91.116.6:4000): protocol: write tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; close tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581606038 (748h41m1.178377062s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1aarPYLumhCAtisqFVYu9iFnZqBHRFMFN2xakzn48kapVp2ae5, address:45.91.116.14:4197): protocol: context deadline exceeded; context deadline exceeded Apr 6 15:41:03.581732624 (748h41m1.178503641s) [seg_pos={0 0}, upload=838298]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.605380574 (748h41m1.202151599s) [upload=838298]: Done uploading segments... Apr 6 15:41:03.605444334 (748h41m1.20221536s) [upload=838298]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.275984567s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [..........XXX...............................................] duration: 488.797055ms trace: https://jaeger.dev.storj.io/trace/28874b4e710a2bcf FAIL (2026-04-06 16:16:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 488.82977ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 1.292309089s trace: https://jaeger.dev.storj.io/trace/449433b6185f502e FAIL (2026-04-06 16:16:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Apr 6 15:40:03.627476193 (748h40m1.224247238s) [upload=838298]: Starting upload Apr 6 15:40:03.627555112 (748h40m1.224326149s) [upload=838298]: Done starting upload Apr 6 15:40:03.627971751 (748h40m1.224742775s) [upload=838298]: Uploading segments... Apr 6 15:40:03.628052062 (748h40m1.224823089s) [upload=838298]: Got next segment. Inline: false Apr 6 15:40:03.628101312 (748h40m1.224872344s) [upload=838298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.628108719 (748h40m1.224879786s) [upload=838298]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.657925511 (748h40m1.254696552s) [seg_pos={0 0}, upload=838298]: Begin upload segment... Apr 6 15:40:03.658006444 (748h40m1.254777507s) [seg_pos={0 0}, upload=838298]: Done begin upload segment. Apr 6 15:40:03.658049451 (748h40m1.254820484s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658127629 (748h40m1.254898659s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658849378 (748h40m1.255620415s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.659549528 (748h40m1.256320563s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660187950 (748h40m1.256958976s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660848819 (748h40m1.257619849s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.661744944 (748h40m1.25851598s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.662381990 (748h40m1.259153021s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.663016005 (748h40m1.25978704s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.665366739 (748h40m1.262137771s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.666327038 (748h40m1.263098052s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.667370344 (748h40m1.264141368s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.668228606 (748h40m1.264999623s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.670639310 (748h40m1.267410325s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.671832750 (748h40m1.268603781s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.672735177 (748h40m1.269506187s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.673761153 (748h40m1.270532166s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.674068775 (748h40m1.27083979s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.768526476 (748h40m1.365297491s) [upload=838298]: Next returned nil segment Apr 6 15:40:03.768546305 (748h40m1.365317356s) [upload=838298]: Waiting for error group managing segments... Apr 6 15:40:04.090183416 (748h40m1.686954485s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111363426 (748h40m1.708134444s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111628225 (748h40m1.708399237s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.123286299 (748h40m1.720057314s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.305166549 (748h40m1.901937573s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.309277855 (748h40m1.906048873s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.310384222 (748h40m1.90715524s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.311558249 (748h40m1.908329268s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.318896832 (748h40m1.915667854s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.319687548 (748h40m1.916458575s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.320573940 (748h40m1.917344962s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.694686967 (748h40m2.291457994s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.695909952 (748h40m2.292680979s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:41:03.579925561 (748h41m1.176696593s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: rpc: dial tcp 45.91.116.13:4181: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580475482 (748h41m1.177246511s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12UvDrJhqPFda5Vu8NEGBVBYXHoXLFWnKHNJqrLLGgfVi14sZsJ, address:45.91.116.5:4331): protocol: write tcp 159.223.143.39:55764->45.91.116.5:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580772045 (748h41m1.177543071s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 159.223.143.39:33530->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581261292 (748h41m1.178032318s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12CuCqHJYbanbuaEMHvtcuQXhvcdahD8eyT2fKDhqtxdSZ6ES4f, address:45.91.116.6:4000): protocol: write tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; close tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581606038 (748h41m1.178377062s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1aarPYLumhCAtisqFVYu9iFnZqBHRFMFN2xakzn48kapVp2ae5, address:45.91.116.14:4197): protocol: context deadline exceeded; context deadline exceeded Apr 6 15:41:03.581732624 (748h41m1.178503641s) [seg_pos={0 0}, upload=838298]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.605380574 (748h41m1.202151599s) [upload=838298]: Done uploading segments... Apr 6 15:41:03.605444334 (748h41m1.20221536s) [upload=838298]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.292399449s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [...........XXX..............................................] duration: 572.475928ms trace: https://jaeger.dev.storj.io/trace/28874b4e710a2bcf FAIL (2026-04-06 16:16:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 572.587834ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 765.263983ms trace: https://jaeger.dev.storj.io/trace/4777dea64937dc1a FAIL (2026-04-06 16:16:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 765.297229ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [............XXX.............................................] duration: 489.213715ms trace: https://jaeger.dev.storj.io/trace/28874b4e710a2bcf FAIL (2026-04-06 16:16:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 489.253862ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 1.238538306s trace: https://jaeger.dev.storj.io/trace/449433b6185f502e FAIL (2026-04-06 16:15:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Apr 6 15:40:03.627476193 (748h40m1.224247238s) [upload=838298]: Starting upload Apr 6 15:40:03.627555112 (748h40m1.224326149s) [upload=838298]: Done starting upload Apr 6 15:40:03.627971751 (748h40m1.224742775s) [upload=838298]: Uploading segments... Apr 6 15:40:03.628052062 (748h40m1.224823089s) [upload=838298]: Got next segment. Inline: false Apr 6 15:40:03.628101312 (748h40m1.224872344s) [upload=838298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.628108719 (748h40m1.224879786s) [upload=838298]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.657925511 (748h40m1.254696552s) [seg_pos={0 0}, upload=838298]: Begin upload segment... Apr 6 15:40:03.658006444 (748h40m1.254777507s) [seg_pos={0 0}, upload=838298]: Done begin upload segment. Apr 6 15:40:03.658049451 (748h40m1.254820484s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658127629 (748h40m1.254898659s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658849378 (748h40m1.255620415s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.659549528 (748h40m1.256320563s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660187950 (748h40m1.256958976s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660848819 (748h40m1.257619849s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.661744944 (748h40m1.25851598s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.662381990 (748h40m1.259153021s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.663016005 (748h40m1.25978704s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.665366739 (748h40m1.262137771s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.666327038 (748h40m1.263098052s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.667370344 (748h40m1.264141368s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.668228606 (748h40m1.264999623s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.670639310 (748h40m1.267410325s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.671832750 (748h40m1.268603781s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.672735177 (748h40m1.269506187s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.673761153 (748h40m1.270532166s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.674068775 (748h40m1.27083979s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.768526476 (748h40m1.365297491s) [upload=838298]: Next returned nil segment Apr 6 15:40:03.768546305 (748h40m1.365317356s) [upload=838298]: Waiting for error group managing segments... Apr 6 15:40:04.090183416 (748h40m1.686954485s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111363426 (748h40m1.708134444s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111628225 (748h40m1.708399237s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.123286299 (748h40m1.720057314s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.305166549 (748h40m1.901937573s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.309277855 (748h40m1.906048873s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.310384222 (748h40m1.90715524s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.311558249 (748h40m1.908329268s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.318896832 (748h40m1.915667854s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.319687548 (748h40m1.916458575s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.320573940 (748h40m1.917344962s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.694686967 (748h40m2.291457994s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.695909952 (748h40m2.292680979s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:41:03.579925561 (748h41m1.176696593s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: rpc: dial tcp 45.91.116.13:4181: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580475482 (748h41m1.177246511s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12UvDrJhqPFda5Vu8NEGBVBYXHoXLFWnKHNJqrLLGgfVi14sZsJ, address:45.91.116.5:4331): protocol: write tcp 159.223.143.39:55764->45.91.116.5:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580772045 (748h41m1.177543071s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 159.223.143.39:33530->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581261292 (748h41m1.178032318s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12CuCqHJYbanbuaEMHvtcuQXhvcdahD8eyT2fKDhqtxdSZ6ES4f, address:45.91.116.6:4000): protocol: write tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; close tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581606038 (748h41m1.178377062s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1aarPYLumhCAtisqFVYu9iFnZqBHRFMFN2xakzn48kapVp2ae5, address:45.91.116.14:4197): protocol: context deadline exceeded; context deadline exceeded Apr 6 15:41:03.581732624 (748h41m1.178503641s) [seg_pos={0 0}, upload=838298]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.605380574 (748h41m1.202151599s) [upload=838298]: Done uploading segments... Apr 6 15:41:03.605444334 (748h41m1.20221536s) [upload=838298]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.238568627s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 598.168117ms trace: https://jaeger.dev.storj.io/trace/4777dea64937dc1a FAIL (2026-04-06 16:15:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 598.206147ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.............XXX............................................] duration: 525.733573ms trace: https://jaeger.dev.storj.io/trace/28874b4e710a2bcf FAIL (2026-04-06 16:14:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 525.764036ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 1.21988139s trace: https://jaeger.dev.storj.io/trace/449433b6185f502e FAIL (2026-04-06 16:14:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Apr 6 15:40:03.627476193 (748h40m1.224247238s) [upload=838298]: Starting upload Apr 6 15:40:03.627555112 (748h40m1.224326149s) [upload=838298]: Done starting upload Apr 6 15:40:03.627971751 (748h40m1.224742775s) [upload=838298]: Uploading segments... Apr 6 15:40:03.628052062 (748h40m1.224823089s) [upload=838298]: Got next segment. Inline: false Apr 6 15:40:03.628101312 (748h40m1.224872344s) [upload=838298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.628108719 (748h40m1.224879786s) [upload=838298]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.657925511 (748h40m1.254696552s) [seg_pos={0 0}, upload=838298]: Begin upload segment... Apr 6 15:40:03.658006444 (748h40m1.254777507s) [seg_pos={0 0}, upload=838298]: Done begin upload segment. Apr 6 15:40:03.658049451 (748h40m1.254820484s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658127629 (748h40m1.254898659s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658849378 (748h40m1.255620415s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.659549528 (748h40m1.256320563s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660187950 (748h40m1.256958976s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660848819 (748h40m1.257619849s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.661744944 (748h40m1.25851598s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.662381990 (748h40m1.259153021s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.663016005 (748h40m1.25978704s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.665366739 (748h40m1.262137771s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.666327038 (748h40m1.263098052s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.667370344 (748h40m1.264141368s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.668228606 (748h40m1.264999623s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.670639310 (748h40m1.267410325s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.671832750 (748h40m1.268603781s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.672735177 (748h40m1.269506187s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.673761153 (748h40m1.270532166s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.674068775 (748h40m1.27083979s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.768526476 (748h40m1.365297491s) [upload=838298]: Next returned nil segment Apr 6 15:40:03.768546305 (748h40m1.365317356s) [upload=838298]: Waiting for error group managing segments... Apr 6 15:40:04.090183416 (748h40m1.686954485s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111363426 (748h40m1.708134444s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111628225 (748h40m1.708399237s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.123286299 (748h40m1.720057314s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.305166549 (748h40m1.901937573s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.309277855 (748h40m1.906048873s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.310384222 (748h40m1.90715524s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.311558249 (748h40m1.908329268s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.318896832 (748h40m1.915667854s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.319687548 (748h40m1.916458575s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.320573940 (748h40m1.917344962s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.694686967 (748h40m2.291457994s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.695909952 (748h40m2.292680979s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:41:03.579925561 (748h41m1.176696593s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: rpc: dial tcp 45.91.116.13:4181: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580475482 (748h41m1.177246511s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12UvDrJhqPFda5Vu8NEGBVBYXHoXLFWnKHNJqrLLGgfVi14sZsJ, address:45.91.116.5:4331): protocol: write tcp 159.223.143.39:55764->45.91.116.5:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580772045 (748h41m1.177543071s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 159.223.143.39:33530->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581261292 (748h41m1.178032318s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12CuCqHJYbanbuaEMHvtcuQXhvcdahD8eyT2fKDhqtxdSZ6ES4f, address:45.91.116.6:4000): protocol: write tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; close tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581606038 (748h41m1.178377062s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1aarPYLumhCAtisqFVYu9iFnZqBHRFMFN2xakzn48kapVp2ae5, address:45.91.116.14:4197): protocol: context deadline exceeded; context deadline exceeded Apr 6 15:41:03.581732624 (748h41m1.178503641s) [seg_pos={0 0}, upload=838298]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.605380574 (748h41m1.202151599s) [upload=838298]: Done uploading segments... Apr 6 15:41:03.605444334 (748h41m1.20221536s) [upload=838298]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.220021307s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 868.93452ms trace: https://jaeger.dev.storj.io/trace/4777dea64937dc1a FAIL (2026-04-06 16:14:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 868.951924ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 1.049761687s trace: https://jaeger.dev.storj.io/trace/449433b6185f502e FAIL (2026-04-06 16:14:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Apr 6 15:40:03.627476193 (748h40m1.224247238s) [upload=838298]: Starting upload Apr 6 15:40:03.627555112 (748h40m1.224326149s) [upload=838298]: Done starting upload Apr 6 15:40:03.627971751 (748h40m1.224742775s) [upload=838298]: Uploading segments... Apr 6 15:40:03.628052062 (748h40m1.224823089s) [upload=838298]: Got next segment. Inline: false Apr 6 15:40:03.628101312 (748h40m1.224872344s) [upload=838298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.628108719 (748h40m1.224879786s) [upload=838298]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.657925511 (748h40m1.254696552s) [seg_pos={0 0}, upload=838298]: Begin upload segment... Apr 6 15:40:03.658006444 (748h40m1.254777507s) [seg_pos={0 0}, upload=838298]: Done begin upload segment. Apr 6 15:40:03.658049451 (748h40m1.254820484s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658127629 (748h40m1.254898659s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658849378 (748h40m1.255620415s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.659549528 (748h40m1.256320563s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660187950 (748h40m1.256958976s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660848819 (748h40m1.257619849s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.661744944 (748h40m1.25851598s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.662381990 (748h40m1.259153021s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.663016005 (748h40m1.25978704s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.665366739 (748h40m1.262137771s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.666327038 (748h40m1.263098052s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.667370344 (748h40m1.264141368s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.668228606 (748h40m1.264999623s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.670639310 (748h40m1.267410325s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.671832750 (748h40m1.268603781s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.672735177 (748h40m1.269506187s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.673761153 (748h40m1.270532166s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.674068775 (748h40m1.27083979s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.768526476 (748h40m1.365297491s) [upload=838298]: Next returned nil segment Apr 6 15:40:03.768546305 (748h40m1.365317356s) [upload=838298]: Waiting for error group managing segments... Apr 6 15:40:04.090183416 (748h40m1.686954485s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111363426 (748h40m1.708134444s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111628225 (748h40m1.708399237s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.123286299 (748h40m1.720057314s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.305166549 (748h40m1.901937573s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.309277855 (748h40m1.906048873s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.310384222 (748h40m1.90715524s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.311558249 (748h40m1.908329268s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.318896832 (748h40m1.915667854s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.319687548 (748h40m1.916458575s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.320573940 (748h40m1.917344962s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.694686967 (748h40m2.291457994s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.695909952 (748h40m2.292680979s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:41:03.579925561 (748h41m1.176696593s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: rpc: dial tcp 45.91.116.13:4181: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580475482 (748h41m1.177246511s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12UvDrJhqPFda5Vu8NEGBVBYXHoXLFWnKHNJqrLLGgfVi14sZsJ, address:45.91.116.5:4331): protocol: write tcp 159.223.143.39:55764->45.91.116.5:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580772045 (748h41m1.177543071s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 159.223.143.39:33530->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581261292 (748h41m1.178032318s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12CuCqHJYbanbuaEMHvtcuQXhvcdahD8eyT2fKDhqtxdSZ6ES4f, address:45.91.116.6:4000): protocol: write tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; close tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581606038 (748h41m1.178377062s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1aarPYLumhCAtisqFVYu9iFnZqBHRFMFN2xakzn48kapVp2ae5, address:45.91.116.14:4197): protocol: context deadline exceeded; context deadline exceeded Apr 6 15:41:03.581732624 (748h41m1.178503641s) [seg_pos={0 0}, upload=838298]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.605380574 (748h41m1.202151599s) [upload=838298]: Done uploading segments... Apr 6 15:41:03.605444334 (748h41m1.20221536s) [upload=838298]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.049877967s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [..............XXX...........................................] duration: 498.303651ms trace: https://jaeger.dev.storj.io/trace/28874b4e710a2bcf FAIL (2026-04-06 16:14:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 498.411649ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 589.267503ms trace: https://jaeger.dev.storj.io/trace/4777dea64937dc1a FAIL (2026-04-06 16:14:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 589.303616ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [...............XXX..........................................] duration: 664.471408ms trace: https://jaeger.dev.storj.io/trace/28874b4e710a2bcf FAIL (2026-04-06 16:13:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 664.498388ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 587.575221ms trace: https://jaeger.dev.storj.io/trace/4777dea64937dc1a FAIL (2026-04-06 16:13:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 587.62645ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [................XXX.........................................] duration: 527.119079ms trace: https://jaeger.dev.storj.io/trace/28874b4e710a2bcf FAIL (2026-04-06 16:13:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 527.146646ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 839.426146ms trace: https://jaeger.dev.storj.io/trace/4777dea64937dc1a FAIL (2026-04-06 16:13:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 839.468361ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 1.206696907s trace: https://jaeger.dev.storj.io/trace/449433b6185f502e FAIL (2026-04-06 16:13:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Apr 6 15:40:03.627476193 (748h40m1.224247238s) [upload=838298]: Starting upload Apr 6 15:40:03.627555112 (748h40m1.224326149s) [upload=838298]: Done starting upload Apr 6 15:40:03.627971751 (748h40m1.224742775s) [upload=838298]: Uploading segments... Apr 6 15:40:03.628052062 (748h40m1.224823089s) [upload=838298]: Got next segment. Inline: false Apr 6 15:40:03.628101312 (748h40m1.224872344s) [upload=838298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.628108719 (748h40m1.224879786s) [upload=838298]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.657925511 (748h40m1.254696552s) [seg_pos={0 0}, upload=838298]: Begin upload segment... Apr 6 15:40:03.658006444 (748h40m1.254777507s) [seg_pos={0 0}, upload=838298]: Done begin upload segment. Apr 6 15:40:03.658049451 (748h40m1.254820484s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658127629 (748h40m1.254898659s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658849378 (748h40m1.255620415s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.659549528 (748h40m1.256320563s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660187950 (748h40m1.256958976s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660848819 (748h40m1.257619849s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.661744944 (748h40m1.25851598s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.662381990 (748h40m1.259153021s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.663016005 (748h40m1.25978704s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.665366739 (748h40m1.262137771s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.666327038 (748h40m1.263098052s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.667370344 (748h40m1.264141368s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.668228606 (748h40m1.264999623s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.670639310 (748h40m1.267410325s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.671832750 (748h40m1.268603781s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.672735177 (748h40m1.269506187s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.673761153 (748h40m1.270532166s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.674068775 (748h40m1.27083979s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.768526476 (748h40m1.365297491s) [upload=838298]: Next returned nil segment Apr 6 15:40:03.768546305 (748h40m1.365317356s) [upload=838298]: Waiting for error group managing segments... Apr 6 15:40:04.090183416 (748h40m1.686954485s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111363426 (748h40m1.708134444s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111628225 (748h40m1.708399237s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.123286299 (748h40m1.720057314s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.305166549 (748h40m1.901937573s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.309277855 (748h40m1.906048873s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.310384222 (748h40m1.90715524s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.311558249 (748h40m1.908329268s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.318896832 (748h40m1.915667854s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.319687548 (748h40m1.916458575s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.320573940 (748h40m1.917344962s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.694686967 (748h40m2.291457994s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.695909952 (748h40m2.292680979s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:41:03.579925561 (748h41m1.176696593s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: rpc: dial tcp 45.91.116.13:4181: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580475482 (748h41m1.177246511s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12UvDrJhqPFda5Vu8NEGBVBYXHoXLFWnKHNJqrLLGgfVi14sZsJ, address:45.91.116.5:4331): protocol: write tcp 159.223.143.39:55764->45.91.116.5:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580772045 (748h41m1.177543071s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 159.223.143.39:33530->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581261292 (748h41m1.178032318s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12CuCqHJYbanbuaEMHvtcuQXhvcdahD8eyT2fKDhqtxdSZ6ES4f, address:45.91.116.6:4000): protocol: write tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; close tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581606038 (748h41m1.178377062s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1aarPYLumhCAtisqFVYu9iFnZqBHRFMFN2xakzn48kapVp2ae5, address:45.91.116.14:4197): protocol: context deadline exceeded; context deadline exceeded Apr 6 15:41:03.581732624 (748h41m1.178503641s) [seg_pos={0 0}, upload=838298]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.605380574 (748h41m1.202151599s) [upload=838298]: Done uploading segments... Apr 6 15:41:03.605444334 (748h41m1.20221536s) [upload=838298]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.20673218s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.................XXX........................................] duration: 703.127661ms trace: https://jaeger.dev.storj.io/trace/28874b4e710a2bcf FAIL (2026-04-06 16:12:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 703.184249ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 1.147416006s trace: https://jaeger.dev.storj.io/trace/449433b6185f502e FAIL (2026-04-06 16:12:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Apr 6 15:40:03.627476193 (748h40m1.224247238s) [upload=838298]: Starting upload Apr 6 15:40:03.627555112 (748h40m1.224326149s) [upload=838298]: Done starting upload Apr 6 15:40:03.627971751 (748h40m1.224742775s) [upload=838298]: Uploading segments... Apr 6 15:40:03.628052062 (748h40m1.224823089s) [upload=838298]: Got next segment. Inline: false Apr 6 15:40:03.628101312 (748h40m1.224872344s) [upload=838298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.628108719 (748h40m1.224879786s) [upload=838298]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.657925511 (748h40m1.254696552s) [seg_pos={0 0}, upload=838298]: Begin upload segment... Apr 6 15:40:03.658006444 (748h40m1.254777507s) [seg_pos={0 0}, upload=838298]: Done begin upload segment. Apr 6 15:40:03.658049451 (748h40m1.254820484s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658127629 (748h40m1.254898659s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658849378 (748h40m1.255620415s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.659549528 (748h40m1.256320563s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660187950 (748h40m1.256958976s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660848819 (748h40m1.257619849s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.661744944 (748h40m1.25851598s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.662381990 (748h40m1.259153021s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.663016005 (748h40m1.25978704s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.665366739 (748h40m1.262137771s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.666327038 (748h40m1.263098052s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.667370344 (748h40m1.264141368s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.668228606 (748h40m1.264999623s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.670639310 (748h40m1.267410325s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.671832750 (748h40m1.268603781s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.672735177 (748h40m1.269506187s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.673761153 (748h40m1.270532166s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.674068775 (748h40m1.27083979s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.768526476 (748h40m1.365297491s) [upload=838298]: Next returned nil segment Apr 6 15:40:03.768546305 (748h40m1.365317356s) [upload=838298]: Waiting for error group managing segments... Apr 6 15:40:04.090183416 (748h40m1.686954485s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111363426 (748h40m1.708134444s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111628225 (748h40m1.708399237s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.123286299 (748h40m1.720057314s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.305166549 (748h40m1.901937573s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.309277855 (748h40m1.906048873s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.310384222 (748h40m1.90715524s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.311558249 (748h40m1.908329268s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.318896832 (748h40m1.915667854s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.319687548 (748h40m1.916458575s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.320573940 (748h40m1.917344962s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.694686967 (748h40m2.291457994s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.695909952 (748h40m2.292680979s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:41:03.579925561 (748h41m1.176696593s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: rpc: dial tcp 45.91.116.13:4181: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580475482 (748h41m1.177246511s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12UvDrJhqPFda5Vu8NEGBVBYXHoXLFWnKHNJqrLLGgfVi14sZsJ, address:45.91.116.5:4331): protocol: write tcp 159.223.143.39:55764->45.91.116.5:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580772045 (748h41m1.177543071s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 159.223.143.39:33530->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581261292 (748h41m1.178032318s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12CuCqHJYbanbuaEMHvtcuQXhvcdahD8eyT2fKDhqtxdSZ6ES4f, address:45.91.116.6:4000): protocol: write tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; close tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581606038 (748h41m1.178377062s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1aarPYLumhCAtisqFVYu9iFnZqBHRFMFN2xakzn48kapVp2ae5, address:45.91.116.14:4197): protocol: context deadline exceeded; context deadline exceeded Apr 6 15:41:03.581732624 (748h41m1.178503641s) [seg_pos={0 0}, upload=838298]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.605380574 (748h41m1.202151599s) [upload=838298]: Done uploading segments... Apr 6 15:41:03.605444334 (748h41m1.20221536s) [upload=838298]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.147520646s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [..................XXX.......................................] duration: 449.743702ms trace: https://jaeger.dev.storj.io/trace/28874b4e710a2bcf FAIL (2026-04-06 16:12:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 449.772375ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 896.970588ms trace: https://jaeger.dev.storj.io/trace/4777dea64937dc1a FAIL (2026-04-06 16:11:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 897.051941ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 1.136251144s trace: https://jaeger.dev.storj.io/trace/449433b6185f502e FAIL (2026-04-06 16:11:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Apr 6 15:40:03.627476193 (748h40m1.224247238s) [upload=838298]: Starting upload Apr 6 15:40:03.627555112 (748h40m1.224326149s) [upload=838298]: Done starting upload Apr 6 15:40:03.627971751 (748h40m1.224742775s) [upload=838298]: Uploading segments... Apr 6 15:40:03.628052062 (748h40m1.224823089s) [upload=838298]: Got next segment. Inline: false Apr 6 15:40:03.628101312 (748h40m1.224872344s) [upload=838298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.628108719 (748h40m1.224879786s) [upload=838298]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.657925511 (748h40m1.254696552s) [seg_pos={0 0}, upload=838298]: Begin upload segment... Apr 6 15:40:03.658006444 (748h40m1.254777507s) [seg_pos={0 0}, upload=838298]: Done begin upload segment. Apr 6 15:40:03.658049451 (748h40m1.254820484s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658127629 (748h40m1.254898659s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658849378 (748h40m1.255620415s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.659549528 (748h40m1.256320563s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660187950 (748h40m1.256958976s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660848819 (748h40m1.257619849s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.661744944 (748h40m1.25851598s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.662381990 (748h40m1.259153021s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.663016005 (748h40m1.25978704s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.665366739 (748h40m1.262137771s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.666327038 (748h40m1.263098052s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.667370344 (748h40m1.264141368s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.668228606 (748h40m1.264999623s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.670639310 (748h40m1.267410325s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.671832750 (748h40m1.268603781s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.672735177 (748h40m1.269506187s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.673761153 (748h40m1.270532166s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.674068775 (748h40m1.27083979s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.768526476 (748h40m1.365297491s) [upload=838298]: Next returned nil segment Apr 6 15:40:03.768546305 (748h40m1.365317356s) [upload=838298]: Waiting for error group managing segments... Apr 6 15:40:04.090183416 (748h40m1.686954485s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111363426 (748h40m1.708134444s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111628225 (748h40m1.708399237s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.123286299 (748h40m1.720057314s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.305166549 (748h40m1.901937573s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.309277855 (748h40m1.906048873s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.310384222 (748h40m1.90715524s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.311558249 (748h40m1.908329268s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.318896832 (748h40m1.915667854s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.319687548 (748h40m1.916458575s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.320573940 (748h40m1.917344962s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.694686967 (748h40m2.291457994s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.695909952 (748h40m2.292680979s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:41:03.579925561 (748h41m1.176696593s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: rpc: dial tcp 45.91.116.13:4181: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580475482 (748h41m1.177246511s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12UvDrJhqPFda5Vu8NEGBVBYXHoXLFWnKHNJqrLLGgfVi14sZsJ, address:45.91.116.5:4331): protocol: write tcp 159.223.143.39:55764->45.91.116.5:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580772045 (748h41m1.177543071s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 159.223.143.39:33530->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581261292 (748h41m1.178032318s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12CuCqHJYbanbuaEMHvtcuQXhvcdahD8eyT2fKDhqtxdSZ6ES4f, address:45.91.116.6:4000): protocol: write tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; close tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581606038 (748h41m1.178377062s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1aarPYLumhCAtisqFVYu9iFnZqBHRFMFN2xakzn48kapVp2ae5, address:45.91.116.14:4197): protocol: context deadline exceeded; context deadline exceeded Apr 6 15:41:03.581732624 (748h41m1.178503641s) [seg_pos={0 0}, upload=838298]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.605380574 (748h41m1.202151599s) [upload=838298]: Done uploading segments... Apr 6 15:41:03.605444334 (748h41m1.20221536s) [upload=838298]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.136282502s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)