Orbiter History

Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure

Orbiter Check History

Showing 50 of 312965 reports (Page 15 of 6260)

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [...................................................XXX......] duration: 503.380525ms trace: https://jaeger.dev.storj.io/trace/28874b4e710a2bcf FAIL (2026-04-06 15:45:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 503.416304ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 1.260639363s trace: https://jaeger.dev.storj.io/trace/449433b6185f502e FAIL (2026-04-06 15:44:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Apr 6 15:40:03.627476193 (748h40m1.224247238s) [upload=838298]: Starting upload Apr 6 15:40:03.627555112 (748h40m1.224326149s) [upload=838298]: Done starting upload Apr 6 15:40:03.627971751 (748h40m1.224742775s) [upload=838298]: Uploading segments... Apr 6 15:40:03.628052062 (748h40m1.224823089s) [upload=838298]: Got next segment. Inline: false Apr 6 15:40:03.628101312 (748h40m1.224872344s) [upload=838298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.628108719 (748h40m1.224879786s) [upload=838298]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.657925511 (748h40m1.254696552s) [seg_pos={0 0}, upload=838298]: Begin upload segment... Apr 6 15:40:03.658006444 (748h40m1.254777507s) [seg_pos={0 0}, upload=838298]: Done begin upload segment. Apr 6 15:40:03.658049451 (748h40m1.254820484s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658127629 (748h40m1.254898659s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658849378 (748h40m1.255620415s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.659549528 (748h40m1.256320563s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660187950 (748h40m1.256958976s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660848819 (748h40m1.257619849s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.661744944 (748h40m1.25851598s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.662381990 (748h40m1.259153021s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.663016005 (748h40m1.25978704s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.665366739 (748h40m1.262137771s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.666327038 (748h40m1.263098052s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.667370344 (748h40m1.264141368s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.668228606 (748h40m1.264999623s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.670639310 (748h40m1.267410325s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.671832750 (748h40m1.268603781s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.672735177 (748h40m1.269506187s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.673761153 (748h40m1.270532166s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.674068775 (748h40m1.27083979s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.768526476 (748h40m1.365297491s) [upload=838298]: Next returned nil segment Apr 6 15:40:03.768546305 (748h40m1.365317356s) [upload=838298]: Waiting for error group managing segments... Apr 6 15:40:04.090183416 (748h40m1.686954485s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111363426 (748h40m1.708134444s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111628225 (748h40m1.708399237s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.123286299 (748h40m1.720057314s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.305166549 (748h40m1.901937573s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.309277855 (748h40m1.906048873s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.310384222 (748h40m1.90715524s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.311558249 (748h40m1.908329268s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.318896832 (748h40m1.915667854s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.319687548 (748h40m1.916458575s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.320573940 (748h40m1.917344962s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.694686967 (748h40m2.291457994s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.695909952 (748h40m2.292680979s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:41:03.579925561 (748h41m1.176696593s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: rpc: dial tcp 45.91.116.13:4181: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580475482 (748h41m1.177246511s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12UvDrJhqPFda5Vu8NEGBVBYXHoXLFWnKHNJqrLLGgfVi14sZsJ, address:45.91.116.5:4331): protocol: write tcp 159.223.143.39:55764->45.91.116.5:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580772045 (748h41m1.177543071s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 159.223.143.39:33530->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581261292 (748h41m1.178032318s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12CuCqHJYbanbuaEMHvtcuQXhvcdahD8eyT2fKDhqtxdSZ6ES4f, address:45.91.116.6:4000): protocol: write tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; close tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581606038 (748h41m1.178377062s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1aarPYLumhCAtisqFVYu9iFnZqBHRFMFN2xakzn48kapVp2ae5, address:45.91.116.14:4197): protocol: context deadline exceeded; context deadline exceeded Apr 6 15:41:03.581732624 (748h41m1.178503641s) [seg_pos={0 0}, upload=838298]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.605380574 (748h41m1.202151599s) [upload=838298]: Done uploading segments... Apr 6 15:41:03.605444334 (748h41m1.20221536s) [upload=838298]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.260731917s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [....................................................XXX.....] duration: 606.918915ms trace: https://jaeger.dev.storj.io/trace/28874b4e710a2bcf FAIL (2026-04-06 15:44:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 606.952417ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 748.852361ms trace: https://jaeger.dev.storj.io/trace/4777dea64937dc1a FAIL (2026-04-06 15:44:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 748.887627ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 1.433638224s trace: https://jaeger.dev.storj.io/trace/449433b6185f502e FAIL (2026-04-06 15:44:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Apr 6 15:40:03.627476193 (748h40m1.224247238s) [upload=838298]: Starting upload Apr 6 15:40:03.627555112 (748h40m1.224326149s) [upload=838298]: Done starting upload Apr 6 15:40:03.627971751 (748h40m1.224742775s) [upload=838298]: Uploading segments... Apr 6 15:40:03.628052062 (748h40m1.224823089s) [upload=838298]: Got next segment. Inline: false Apr 6 15:40:03.628101312 (748h40m1.224872344s) [upload=838298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.628108719 (748h40m1.224879786s) [upload=838298]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.657925511 (748h40m1.254696552s) [seg_pos={0 0}, upload=838298]: Begin upload segment... Apr 6 15:40:03.658006444 (748h40m1.254777507s) [seg_pos={0 0}, upload=838298]: Done begin upload segment. Apr 6 15:40:03.658049451 (748h40m1.254820484s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658127629 (748h40m1.254898659s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658849378 (748h40m1.255620415s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.659549528 (748h40m1.256320563s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660187950 (748h40m1.256958976s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660848819 (748h40m1.257619849s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.661744944 (748h40m1.25851598s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.662381990 (748h40m1.259153021s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.663016005 (748h40m1.25978704s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.665366739 (748h40m1.262137771s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.666327038 (748h40m1.263098052s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.667370344 (748h40m1.264141368s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.668228606 (748h40m1.264999623s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.670639310 (748h40m1.267410325s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.671832750 (748h40m1.268603781s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.672735177 (748h40m1.269506187s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.673761153 (748h40m1.270532166s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.674068775 (748h40m1.27083979s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.768526476 (748h40m1.365297491s) [upload=838298]: Next returned nil segment Apr 6 15:40:03.768546305 (748h40m1.365317356s) [upload=838298]: Waiting for error group managing segments... Apr 6 15:40:04.090183416 (748h40m1.686954485s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111363426 (748h40m1.708134444s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111628225 (748h40m1.708399237s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.123286299 (748h40m1.720057314s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.305166549 (748h40m1.901937573s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.309277855 (748h40m1.906048873s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.310384222 (748h40m1.90715524s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.311558249 (748h40m1.908329268s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.318896832 (748h40m1.915667854s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.319687548 (748h40m1.916458575s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.320573940 (748h40m1.917344962s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.694686967 (748h40m2.291457994s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.695909952 (748h40m2.292680979s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:41:03.579925561 (748h41m1.176696593s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: rpc: dial tcp 45.91.116.13:4181: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580475482 (748h41m1.177246511s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12UvDrJhqPFda5Vu8NEGBVBYXHoXLFWnKHNJqrLLGgfVi14sZsJ, address:45.91.116.5:4331): protocol: write tcp 159.223.143.39:55764->45.91.116.5:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580772045 (748h41m1.177543071s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 159.223.143.39:33530->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581261292 (748h41m1.178032318s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12CuCqHJYbanbuaEMHvtcuQXhvcdahD8eyT2fKDhqtxdSZ6ES4f, address:45.91.116.6:4000): protocol: write tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; close tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581606038 (748h41m1.178377062s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1aarPYLumhCAtisqFVYu9iFnZqBHRFMFN2xakzn48kapVp2ae5, address:45.91.116.14:4197): protocol: context deadline exceeded; context deadline exceeded Apr 6 15:41:03.581732624 (748h41m1.178503641s) [seg_pos={0 0}, upload=838298]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.605380574 (748h41m1.202151599s) [upload=838298]: Done uploading segments... Apr 6 15:41:03.605444334 (748h41m1.20221536s) [upload=838298]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.433664927s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 743.548556ms trace: https://jaeger.dev.storj.io/trace/4777dea64937dc1a FAIL (2026-04-06 15:43:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 743.587775ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.....................................................XXX....] duration: 567.245721ms trace: https://jaeger.dev.storj.io/trace/28874b4e710a2bcf FAIL (2026-04-06 15:43:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 567.273568ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [......................................................XXX...] duration: 831.520532ms trace: https://jaeger.dev.storj.io/trace/28874b4e710a2bcf FAIL (2026-04-06 15:43:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 831.548943ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 710.090206ms trace: https://jaeger.dev.storj.io/trace/4777dea64937dc1a FAIL (2026-04-06 15:43:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 710.852223ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 1.416530783s trace: https://jaeger.dev.storj.io/trace/449433b6185f502e FAIL (2026-04-06 15:43:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Apr 6 15:40:03.627476193 (748h40m1.224247238s) [upload=838298]: Starting upload Apr 6 15:40:03.627555112 (748h40m1.224326149s) [upload=838298]: Done starting upload Apr 6 15:40:03.627971751 (748h40m1.224742775s) [upload=838298]: Uploading segments... Apr 6 15:40:03.628052062 (748h40m1.224823089s) [upload=838298]: Got next segment. Inline: false Apr 6 15:40:03.628101312 (748h40m1.224872344s) [upload=838298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.628108719 (748h40m1.224879786s) [upload=838298]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.657925511 (748h40m1.254696552s) [seg_pos={0 0}, upload=838298]: Begin upload segment... Apr 6 15:40:03.658006444 (748h40m1.254777507s) [seg_pos={0 0}, upload=838298]: Done begin upload segment. Apr 6 15:40:03.658049451 (748h40m1.254820484s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658127629 (748h40m1.254898659s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658849378 (748h40m1.255620415s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.659549528 (748h40m1.256320563s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660187950 (748h40m1.256958976s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660848819 (748h40m1.257619849s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.661744944 (748h40m1.25851598s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.662381990 (748h40m1.259153021s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.663016005 (748h40m1.25978704s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.665366739 (748h40m1.262137771s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.666327038 (748h40m1.263098052s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.667370344 (748h40m1.264141368s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.668228606 (748h40m1.264999623s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.670639310 (748h40m1.267410325s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.671832750 (748h40m1.268603781s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.672735177 (748h40m1.269506187s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.673761153 (748h40m1.270532166s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.674068775 (748h40m1.27083979s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.768526476 (748h40m1.365297491s) [upload=838298]: Next returned nil segment Apr 6 15:40:03.768546305 (748h40m1.365317356s) [upload=838298]: Waiting for error group managing segments... Apr 6 15:40:04.090183416 (748h40m1.686954485s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111363426 (748h40m1.708134444s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111628225 (748h40m1.708399237s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.123286299 (748h40m1.720057314s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.305166549 (748h40m1.901937573s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.309277855 (748h40m1.906048873s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.310384222 (748h40m1.90715524s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.311558249 (748h40m1.908329268s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.318896832 (748h40m1.915667854s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.319687548 (748h40m1.916458575s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.320573940 (748h40m1.917344962s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.694686967 (748h40m2.291457994s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.695909952 (748h40m2.292680979s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:41:03.579925561 (748h41m1.176696593s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: rpc: dial tcp 45.91.116.13:4181: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580475482 (748h41m1.177246511s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12UvDrJhqPFda5Vu8NEGBVBYXHoXLFWnKHNJqrLLGgfVi14sZsJ, address:45.91.116.5:4331): protocol: write tcp 159.223.143.39:55764->45.91.116.5:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580772045 (748h41m1.177543071s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 159.223.143.39:33530->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581261292 (748h41m1.178032318s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12CuCqHJYbanbuaEMHvtcuQXhvcdahD8eyT2fKDhqtxdSZ6ES4f, address:45.91.116.6:4000): protocol: write tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; close tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581606038 (748h41m1.178377062s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1aarPYLumhCAtisqFVYu9iFnZqBHRFMFN2xakzn48kapVp2ae5, address:45.91.116.14:4197): protocol: context deadline exceeded; context deadline exceeded Apr 6 15:41:03.581732624 (748h41m1.178503641s) [seg_pos={0 0}, upload=838298]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.605380574 (748h41m1.202151599s) [upload=838298]: Done uploading segments... Apr 6 15:41:03.605444334 (748h41m1.20221536s) [upload=838298]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.416560383s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.......................................................XXX..] duration: 481.930704ms trace: https://jaeger.dev.storj.io/trace/28874b4e710a2bcf FAIL (2026-04-06 15:42:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 482.02529ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 907.971904ms trace: https://jaeger.dev.storj.io/trace/4777dea64937dc1a FAIL (2026-04-06 15:42:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 908.013947ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 2.530970587s trace: https://jaeger.dev.storj.io/trace/449433b6185f502e FAIL (2026-04-06 15:42:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Apr 6 15:40:03.627476193 (748h40m1.224247238s) [upload=838298]: Starting upload Apr 6 15:40:03.627555112 (748h40m1.224326149s) [upload=838298]: Done starting upload Apr 6 15:40:03.627971751 (748h40m1.224742775s) [upload=838298]: Uploading segments... Apr 6 15:40:03.628052062 (748h40m1.224823089s) [upload=838298]: Got next segment. Inline: false Apr 6 15:40:03.628101312 (748h40m1.224872344s) [upload=838298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.628108719 (748h40m1.224879786s) [upload=838298]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.657925511 (748h40m1.254696552s) [seg_pos={0 0}, upload=838298]: Begin upload segment... Apr 6 15:40:03.658006444 (748h40m1.254777507s) [seg_pos={0 0}, upload=838298]: Done begin upload segment. Apr 6 15:40:03.658049451 (748h40m1.254820484s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658127629 (748h40m1.254898659s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658849378 (748h40m1.255620415s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.659549528 (748h40m1.256320563s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660187950 (748h40m1.256958976s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660848819 (748h40m1.257619849s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.661744944 (748h40m1.25851598s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.662381990 (748h40m1.259153021s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.663016005 (748h40m1.25978704s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.665366739 (748h40m1.262137771s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.666327038 (748h40m1.263098052s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.667370344 (748h40m1.264141368s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.668228606 (748h40m1.264999623s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.670639310 (748h40m1.267410325s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.671832750 (748h40m1.268603781s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.672735177 (748h40m1.269506187s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.673761153 (748h40m1.270532166s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.674068775 (748h40m1.27083979s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.768526476 (748h40m1.365297491s) [upload=838298]: Next returned nil segment Apr 6 15:40:03.768546305 (748h40m1.365317356s) [upload=838298]: Waiting for error group managing segments... Apr 6 15:40:04.090183416 (748h40m1.686954485s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111363426 (748h40m1.708134444s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111628225 (748h40m1.708399237s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.123286299 (748h40m1.720057314s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.305166549 (748h40m1.901937573s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.309277855 (748h40m1.906048873s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.310384222 (748h40m1.90715524s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.311558249 (748h40m1.908329268s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.318896832 (748h40m1.915667854s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.319687548 (748h40m1.916458575s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.320573940 (748h40m1.917344962s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.694686967 (748h40m2.291457994s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.695909952 (748h40m2.292680979s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:41:03.579925561 (748h41m1.176696593s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: rpc: dial tcp 45.91.116.13:4181: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580475482 (748h41m1.177246511s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12UvDrJhqPFda5Vu8NEGBVBYXHoXLFWnKHNJqrLLGgfVi14sZsJ, address:45.91.116.5:4331): protocol: write tcp 159.223.143.39:55764->45.91.116.5:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580772045 (748h41m1.177543071s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 159.223.143.39:33530->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581261292 (748h41m1.178032318s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12CuCqHJYbanbuaEMHvtcuQXhvcdahD8eyT2fKDhqtxdSZ6ES4f, address:45.91.116.6:4000): protocol: write tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; close tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581606038 (748h41m1.178377062s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1aarPYLumhCAtisqFVYu9iFnZqBHRFMFN2xakzn48kapVp2ae5, address:45.91.116.14:4197): protocol: context deadline exceeded; context deadline exceeded Apr 6 15:41:03.581732624 (748h41m1.178503641s) [seg_pos={0 0}, upload=838298]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.605380574 (748h41m1.202151599s) [upload=838298]: Done uploading segments... Apr 6 15:41:03.605444334 (748h41m1.20221536s) [upload=838298]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.531031454s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [........................................................XXX.] duration: 1.029721179s trace: https://jaeger.dev.storj.io/trace/28874b4e710a2bcf FAIL (2026-04-06 15:42:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.029751874s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 2.64533695s trace: https://jaeger.dev.storj.io/trace/4777dea64937dc1a FAIL (2026-04-06 15:41:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.645411427s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 45.761189857s trace: https://jaeger.dev.storj.io/trace/4777dea64937dc1a FAIL (2026-04-06 15:41:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 45.761214336s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 2.163864897s trace: https://jaeger.dev.storj.io/trace/449433b6185f502e FAIL (2026-04-06 15:41:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Apr 6 15:40:03.627476193 (748h40m1.224247238s) [upload=838298]: Starting upload Apr 6 15:40:03.627555112 (748h40m1.224326149s) [upload=838298]: Done starting upload Apr 6 15:40:03.627971751 (748h40m1.224742775s) [upload=838298]: Uploading segments... Apr 6 15:40:03.628052062 (748h40m1.224823089s) [upload=838298]: Got next segment. Inline: false Apr 6 15:40:03.628101312 (748h40m1.224872344s) [upload=838298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.628108719 (748h40m1.224879786s) [upload=838298]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.657925511 (748h40m1.254696552s) [seg_pos={0 0}, upload=838298]: Begin upload segment... Apr 6 15:40:03.658006444 (748h40m1.254777507s) [seg_pos={0 0}, upload=838298]: Done begin upload segment. Apr 6 15:40:03.658049451 (748h40m1.254820484s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658127629 (748h40m1.254898659s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658849378 (748h40m1.255620415s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.659549528 (748h40m1.256320563s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660187950 (748h40m1.256958976s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660848819 (748h40m1.257619849s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.661744944 (748h40m1.25851598s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.662381990 (748h40m1.259153021s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.663016005 (748h40m1.25978704s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.665366739 (748h40m1.262137771s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.666327038 (748h40m1.263098052s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.667370344 (748h40m1.264141368s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.668228606 (748h40m1.264999623s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.670639310 (748h40m1.267410325s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.671832750 (748h40m1.268603781s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.672735177 (748h40m1.269506187s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.673761153 (748h40m1.270532166s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.674068775 (748h40m1.27083979s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.768526476 (748h40m1.365297491s) [upload=838298]: Next returned nil segment Apr 6 15:40:03.768546305 (748h40m1.365317356s) [upload=838298]: Waiting for error group managing segments... Apr 6 15:40:04.090183416 (748h40m1.686954485s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111363426 (748h40m1.708134444s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111628225 (748h40m1.708399237s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.123286299 (748h40m1.720057314s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.305166549 (748h40m1.901937573s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.309277855 (748h40m1.906048873s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.310384222 (748h40m1.90715524s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.311558249 (748h40m1.908329268s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.318896832 (748h40m1.915667854s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.319687548 (748h40m1.916458575s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.320573940 (748h40m1.917344962s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.694686967 (748h40m2.291457994s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.695909952 (748h40m2.292680979s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:41:03.579925561 (748h41m1.176696593s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: rpc: dial tcp 45.91.116.13:4181: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580475482 (748h41m1.177246511s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12UvDrJhqPFda5Vu8NEGBVBYXHoXLFWnKHNJqrLLGgfVi14sZsJ, address:45.91.116.5:4331): protocol: write tcp 159.223.143.39:55764->45.91.116.5:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580772045 (748h41m1.177543071s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 159.223.143.39:33530->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581261292 (748h41m1.178032318s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12CuCqHJYbanbuaEMHvtcuQXhvcdahD8eyT2fKDhqtxdSZ6ES4f, address:45.91.116.6:4000): protocol: write tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; close tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581606038 (748h41m1.178377062s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1aarPYLumhCAtisqFVYu9iFnZqBHRFMFN2xakzn48kapVp2ae5, address:45.91.116.14:4197): protocol: context deadline exceeded; context deadline exceeded Apr 6 15:41:03.581732624 (748h41m1.178503641s) [seg_pos={0 0}, upload=838298]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.605380574 (748h41m1.202151599s) [upload=838298]: Done uploading segments... Apr 6 15:41:03.605444334 (748h41m1.20221536s) [upload=838298]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.163899493s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.028379807s trace: https://jaeger.dev.storj.io/trace/449433b6185f502e FAIL (2026-04-06 15:41:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Apr 6 15:40:03.627476193 (748h40m1.224247238s) [upload=838298]: Starting upload Apr 6 15:40:03.627555112 (748h40m1.224326149s) [upload=838298]: Done starting upload Apr 6 15:40:03.627971751 (748h40m1.224742775s) [upload=838298]: Uploading segments... Apr 6 15:40:03.628052062 (748h40m1.224823089s) [upload=838298]: Got next segment. Inline: false Apr 6 15:40:03.628101312 (748h40m1.224872344s) [upload=838298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 15:40:03.628108719 (748h40m1.224879786s) [upload=838298]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 15:40:03.657925511 (748h40m1.254696552s) [seg_pos={0 0}, upload=838298]: Begin upload segment... Apr 6 15:40:03.658006444 (748h40m1.254777507s) [seg_pos={0 0}, upload=838298]: Done begin upload segment. Apr 6 15:40:03.658049451 (748h40m1.254820484s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658127629 (748h40m1.254898659s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.658849378 (748h40m1.255620415s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.659549528 (748h40m1.256320563s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660187950 (748h40m1.256958976s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.660848819 (748h40m1.257619849s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.661744944 (748h40m1.25851598s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.662381990 (748h40m1.259153021s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.663016005 (748h40m1.25978704s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.665366739 (748h40m1.262137771s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.666327038 (748h40m1.263098052s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.667370344 (748h40m1.264141368s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.668228606 (748h40m1.264999623s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.670639310 (748h40m1.267410325s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.671832750 (748h40m1.268603781s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.672735177 (748h40m1.269506187s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.673761153 (748h40m1.270532166s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.674068775 (748h40m1.27083979s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Uploading piece... Apr 6 15:40:03.768526476 (748h40m1.365297491s) [upload=838298]: Next returned nil segment Apr 6 15:40:03.768546305 (748h40m1.365317356s) [upload=838298]: Waiting for error group managing segments... Apr 6 15:40:04.090183416 (748h40m1.686954485s) [noise=true , address=167.17.72.71:4797 , piece_id=SCBKHW7H7YFLYZZFVT6JSBT6XFUSKQYOW3VRN275AJ7MN4YI4AOA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111363426 (748h40m1.708134444s) [noise=true , address=167.17.72.69:4734 , piece_id=JPHKDBK4IEMSZRNHRSDTGMSDNP6VFJZM7FLVUZOBLPZEESBFYIOQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.111628225 (748h40m1.708399237s) [noise=true , address=167.17.72.70:4754 , piece_id=WTUQ433XRWNDGHLG7FXULE77N3EQZI3L2RAZGZX24TQJAAPM7WNQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.123286299 (748h40m1.720057314s) [noise=true , address=167.17.72.68:4644 , piece_id=2SUMWQO5FV3WABHHGVLOHKNHY6XTEWTBOK7H2ZO5NKB5TRBTHVTA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.305166549 (748h40m1.901937573s) [noise=true , address=193.149.164.68:4065 , piece_id=ZUBZPNKM3WD6EC22MBDC54N6PDBMXPR3IFNWUZBOXJPDBELHTCRQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.309277855 (748h40m1.906048873s) [noise=true , address=193.149.164.71:4287 , piece_id=SZ7LQNXTEHH74XPWRFSNGS6LIM44IF7HFABJJ74EKHCSM5TOX63Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.310384222 (748h40m1.90715524s) [noise=true , address=193.149.164.70:4134 , piece_id=ADFW67E4S7IGQQ6GNB7R7JU43YVLSTEOOBXF5WGIN64OLBYI6S7A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.311558249 (748h40m1.908329268s) [noise=true , address=64.49.13.170:4723 , piece_id=VZG4UM2B4YUC7NYFCITOADGLWDBPEYGHJUHUCG5TTMW2OXNGDZ3Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.318896832 (748h40m1.915667854s) [noise=true , address=193.149.164.69:4073 , piece_id=X6LKMM4OEB36MKVK62Z2SU2FQ6QMMOZNO7V6AW23CSEX6QWWPL2A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.319687548 (748h40m1.916458575s) [noise=true , address=64.49.13.139:4471 , piece_id=7XPAYWANYD2ULF6VKQ62CDFV4R3GWZSLO3PXBU3OHIMTERSVGJ6Q, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.320573940 (748h40m1.917344962s) [noise=true , address=193.149.164.67:4003 , piece_id=BYZKHKH5VCAXFXQAKFUNQ5AUPXXGMOR4S6N46ZYCEWDTHXLIB2RA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.694686967 (748h40m2.291457994s) [noise=true , address=145.79.227.30:4773 , piece_id=EWG62WQ5HGNTKWAQUDO2JMXPLMGCVOINHNB32LKQG4FZZRYHWG4A, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:40:04.695909952 (748h40m2.292680979s) [noise=true , address=145.79.227.34:4911 , piece_id=3XYN5FDW46S63WCK72XALIRL5OXIJQ24WPPD6IZLE53XDPKN2PEQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: <nil> Apr 6 15:41:03.579925561 (748h41m1.176696593s) [noise=true , address=45.91.116.13:4181 , piece_id=IKW65PKAUL2VH4NAK75PROH3UVR3S2WXX4U66JJKCEM3AYMEB4SA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1h7LXGgUieZvwdvxcRGCLGSQVVV2PjFuEaze7ccG1hJLNB2aQ, address:45.91.116.13:4181): protocol: rpc: dial tcp 45.91.116.13:4181: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580475482 (748h41m1.177246511s) [noise=true , address=45.91.116.5:4331 , piece_id=MWV7QE3D7YQYLH6JXFB3FPLZ7JDCQ4QFJJTQD3V4RRBZCZSAVNIA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12UvDrJhqPFda5Vu8NEGBVBYXHoXLFWnKHNJqrLLGgfVi14sZsJ, address:45.91.116.5:4331): protocol: write tcp 159.223.143.39:55764->45.91.116.5:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.580772045 (748h41m1.177543071s) [noise=true , address=45.91.116.7:4025 , piece_id=VCTH56NPYPTY6SWJMBS3IKUOBA5J7NJYN7YXNVNFXBHVZVYE77LQ, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 159.223.143.39:33530->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581261292 (748h41m1.178032318s) [noise=true , address=45.91.116.6:4000 , piece_id=XP2BZQO5FXULIPVB4QJWKZUVA2MZVWJ3GCUIR6YQDI5S2VVAMQVA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:12CuCqHJYbanbuaEMHvtcuQXhvcdahD8eyT2fKDhqtxdSZ6ES4f, address:45.91.116.6:4000): protocol: write tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; close tcp 159.223.143.39:48362->45.91.116.6:4000: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 15:41:03.581606038 (748h41m1.178377062s) [noise=true , address=45.91.116.14:4197 , piece_id=LF6Z376GEE7C6IBOUIITPI4GTANECFOUL4RZ2GMBS4XVVQDRWJGA, seg_pos={0 0}, upload=838298]: Done uploading piece. err: ecclient: upload failed (node:1aarPYLumhCAtisqFVYu9iFnZqBHRFMFN2xakzn48kapVp2ae5, address:45.91.116.14:4197): protocol: context deadline exceeded; context deadline exceeded Apr 6 15:41:03.581732624 (748h41m1.178503641s) [seg_pos={0 0}, upload=838298]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 15:41:03.605380574 (748h41m1.202151599s) [upload=838298]: Done uploading segments... Apr 6 15:41:03.605444334 (748h41m1.20221536s) [upload=838298]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.028422588s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.........................................................XXX] duration: 5.002614009s trace: https://jaeger.dev.storj.io/trace/28874b4e710a2bcf FAIL (2026-04-06 15:41:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.00265671s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.000996248s trace: https://jaeger.dev.storj.io/trace/4777dea64937dc1a FAIL (2026-04-06 15:40:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001027708s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..........................................................XX] duration: 5.001972687s trace: https://jaeger.dev.storj.io/trace/746e64e14f211a7c FAIL (2026-04-06 15:40:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.002002361s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................................................X] duration: 5.001426537s trace: https://jaeger.dev.storj.io/trace/501f9a50526e1d4 FAIL (2026-04-06 15:39:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001452129s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 1.224193772s trace: https://jaeger.dev.storj.io/trace/7e44fa01de290116 FAIL (2026-04-06 11:27:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== loading segments... Apr 6 10:39:12.293175658 (743h39m9.88994668s) [upload=832686]: Got next segment. Inline: false Apr 6 10:39:12.293210466 (743h39m9.889981489s) [upload=832686]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 10:39:12.293215002 (743h39m9.889986025s) [upload=832686]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 10:39:12.319174409 (743h39m9.915945433s) [seg_pos={0 0}, upload=832686]: Begin upload segment... Apr 6 10:39:12.319264769 (743h39m9.916035785s) [seg_pos={0 0}, upload=832686]: Done begin upload segment. Apr 6 10:39:12.319310748 (743h39m9.91608178s) [noise=true , address=45.91.116.13:4166 , piece_id=CNVYP6YUT7E3VUCCSUR2ZWCDYU6QMOZL5GPZAMPQ5P2BVSD6HUDQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.319344730 (743h39m9.916115762s) [noise=true , address=45.91.116.6:4987 , piece_id=N2OUZHWZMNVIMAPTOHLIIT6NK5N3PVSHS6OX4ZGYSRUJIS5X3CDQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.319839277 (743h39m9.916610309s) [noise=true , address=193.149.164.67:4008 , piece_id=SJM4PFZHQJW7I5C3XWXIIAJF7DBXPQQYGHTB22AB7JKMG6VN7QZA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320290481 (743h39m9.917061503s) [noise=true , address=167.17.72.69:4734 , piece_id=I7HV3YRWA4IH7VE2I7SQFA42VHSYJVZCKA2KP4XQ4KSRJ6S7KPLA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320450536 (743h39m9.917221562s) [noise=true , address=45.91.116.9:4270 , piece_id=UZYXLU235ANTBCMIIQDUHM6LZUKAS64FUBJIY2NXGYTIAO776KFQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320733701 (743h39m9.917504727s) [noise=true , address=193.149.164.143:4694 , piece_id=GG3FQN4YC5QQVGFCFYWTZWOZPFQO5NL2XE4FJCSUKUCTY4L5EWMA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320988084 (743h39m9.917759097s) [noise=true , address=45.91.116.12:4257 , piece_id=GW7MFYT77HZYDY263WMUIABDAELV37MBKMBS57FFSFXCIUJ5JN7A, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321148099 (743h39m9.917919124s) [noise=true , address=145.79.227.37:4582 , piece_id=QFSMYYEUDOO5JURWQHBLBRLCOXYSP4M2OR4OBWKIZ5LCCXPXEJCQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321494813 (743h39m9.918265836s) [noise=true , address=45.91.116.11:4142 , piece_id=YAVLJVKDZWHWPN572Y5TX55ONWE2IDXVA6ABUF6KD7YRWCWR3YSQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321531682 (743h39m9.918302714s) [noise=true , address=193.149.164.68:4055 , piece_id=ARLLKHKO3AQJG52QH6EWHE55IKPQIEYCIIOR4RMIMU55TGYXYGKQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321879088 (743h39m9.918650182s) [noise=true , address=193.149.164.69:4104 , piece_id=Q3VXWW73OASMNUYH77EZBCQU5GAGPG3X3TKRLED43F3R6MZYZ2DQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322027388 (743h39m9.918798417s) [noise=true , address=193.149.164.66:4985 , piece_id=A6YQDCH7PTS6CP3IFWYMQ3HJGKFX6OSYQBATKUAIOXELHAEE5WBA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322241446 (743h39m9.919012475s) [noise=true , address=145.79.227.33:4878 , piece_id=YAEEULYYAHZP3RM6YAE55LC7S7DQLVV6JWIGYCY64GF6ITHJRQSQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322546472 (743h39m9.919317502s) [noise=true , address=167.17.72.70:4761 , piece_id=WFSQTX3PFW77TZQCSIUSHSD3LH74XTFKLYUMDNYG7MFERLUCSZHA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322583845 (743h39m9.919354868s) [noise=true , address=145.79.227.41:4116 , piece_id=E43SWVMVKWDULRK4YV6SDVJBCHWSAHSFRNACXSZZZQM2IU5JJ4KA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322937532 (743h39m9.919708565s) [noise=true , address=193.149.164.70:4105 , piece_id=7CKN7FHVF6OXY6ZPFZPM4DYNDOJNHGGF3PF3VFM7V2FSTAURFGWA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.323051576 (743h39m9.919822591s) [noise=true , address=77.90.31.17:4775 , piece_id=CFR7LFL2JZCVX4N7X2QQPJVJLZDSD6NZCNLNLSUJKU4DIZ7BBL3Q, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.323474343 (743h39m9.920245373s) [noise=true , address=167.17.72.68:4665 , piece_id=H7Y37VGEKXJGRPEZ2G4NG6WXTNNJ6YQMQI4NRUZYJTWMWIVF246Q, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.386273803 (743h39m9.983044815s) [upload=832686]: Next returned nil segment Apr 6 10:39:12.386288014 (743h39m9.983059037s) [upload=832686]: Waiting for error group managing segments... Apr 6 10:39:12.392899406 (743h39m9.989670418s) [noise=true , address=77.90.31.17:4775 , piece_id=CFR7LFL2JZCVX4N7X2QQPJVJLZDSD6NZCNLNLSUJKU4DIZ7BBL3Q, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.393163291 (743h39m9.989934315s) [noise=true , address=145.79.227.41:4116 , piece_id=E43SWVMVKWDULRK4YV6SDVJBCHWSAHSFRNACXSZZZQM2IU5JJ4KA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.667307989 (743h39m10.264079022s) [noise=true , address=167.17.72.68:4665 , piece_id=H7Y37VGEKXJGRPEZ2G4NG6WXTNNJ6YQMQI4NRUZYJTWMWIVF246Q, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.714497903 (743h39m10.311268937s) [noise=true , address=167.17.72.69:4734 , piece_id=I7HV3YRWA4IH7VE2I7SQFA42VHSYJVZCKA2KP4XQ4KSRJ6S7KPLA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.734199134 (743h39m10.330970147s) [noise=true , address=167.17.72.70:4761 , piece_id=WFSQTX3PFW77TZQCSIUSHSD3LH74XTFKLYUMDNYG7MFERLUCSZHA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.896708412 (743h39m10.493479439s) [noise=true , address=193.149.164.69:4104 , piece_id=Q3VXWW73OASMNUYH77EZBCQU5GAGPG3X3TKRLED43F3R6MZYZ2DQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.899988929 (743h39m10.496759964s) [noise=true , address=193.149.164.67:4008 , piece_id=SJM4PFZHQJW7I5C3XWXIIAJF7DBXPQQYGHTB22AB7JKMG6VN7QZA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.908433760 (743h39m10.505204783s) [noise=true , address=193.149.164.70:4105 , piece_id=7CKN7FHVF6OXY6ZPFZPM4DYNDOJNHGGF3PF3VFM7V2FSTAURFGWA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.916130090 (743h39m10.512901122s) [noise=true , address=193.149.164.68:4055 , piece_id=ARLLKHKO3AQJG52QH6EWHE55IKPQIEYCIIOR4RMIMU55TGYXYGKQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.956247905 (743h39m10.553018928s) [noise=true , address=193.149.164.66:4985 , piece_id=A6YQDCH7PTS6CP3IFWYMQ3HJGKFX6OSYQBATKUAIOXELHAEE5WBA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:13.000961275 (743h39m10.597732326s) [noise=true , address=193.149.164.143:4694 , piece_id=GG3FQN4YC5QQVGFCFYWTZWOZPFQO5NL2XE4FJCSUKUCTY4L5EWMA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:13.304880608 (743h39m10.90165162s) [noise=true , address=145.79.227.33:4878 , piece_id=YAEEULYYAHZP3RM6YAE55LC7S7DQLVV6JWIGYCY64GF6ITHJRQSQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:13.307202541 (743h39m10.903973568s) [noise=true , address=145.79.227.37:4582 , piece_id=QFSMYYEUDOO5JURWQHBLBRLCOXYSP4M2OR4OBWKIZ5LCCXPXEJCQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:40:12.257884464 (743h40m9.854655494s) [noise=true , address=45.91.116.12:4257 , piece_id=GW7MFYT77HZYDY263WMUIABDAELV37MBKMBS57FFSFXCIUJ5JN7A, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:12ZQ5a1TwxdeuJ6UwsVm9VsWnpZEBKC6N5yYUjyNTBkriSjvYvE, address:45.91.116.12:4257): protocol: rpc: dial tcp 45.91.116.12:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258122811 (743h40m9.854893836s) [noise=true , address=45.91.116.9:4270 , piece_id=UZYXLU235ANTBCMIIQDUHM6LZUKAS64FUBJIY2NXGYTIAO776KFQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:1Vr5WmL6Gem29LQTsCwQNFuqw6j4b1nj35AgbUWDA2Lyxf43YL, address:45.91.116.9:4270): protocol: read tcp 159.223.143.39:53356->45.91.116.9:4270: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258358693 (743h40m9.855129725s) [noise=true , address=45.91.116.6:4987 , piece_id=N2OUZHWZMNVIMAPTOHLIIT6NK5N3PVSHS6OX4ZGYSRUJIS5X3CDQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:12vPKcqLhopcpFxZ1LXwD8JXPLJB8a6HKfGnoyXsvKaL2oWUNao, address:45.91.116.6:4987): protocol: context deadline exceeded; context deadline exceeded Apr 6 10:40:12.258544102 (743h40m9.855315131s) [noise=true , address=45.91.116.11:4142 , piece_id=YAVLJVKDZWHWPN572Y5TX55ONWE2IDXVA6ABUF6KD7YRWCWR3YSQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:135mYyE4Gy1gcCXSocnb41e9u4z4MpqMjodmaRnrNs9AGS1YcX, address:45.91.116.11:4142): protocol: write tcp 159.223.143.39:34030->45.91.116.11:4142: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258881055 (743h40m9.855652086s) [noise=true , address=45.91.116.13:4166 , piece_id=CNVYP6YUT7E3VUCCSUR2ZWCDYU6QMOZL5GPZAMPQ5P2BVSD6HUDQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:12ajyozgSuf68kn8sUuEmuSfYCcxzU8Cwmg1EXVt9WV6dKSopy7, address:45.91.116.13:4166): protocol: write tcp 159.223.143.39:45960->45.91.116.13:4166: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258934014 (743h40m9.855705039s) [seg_pos={0 0}, upload=832686]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 10:40:12.278983295 (743h40m9.875754318s) [upload=832686]: Done uploading segments... Apr 6 10:40:12.279076909 (743h40m9.875847939s) [upload=832686]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.224232023s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 1.334705789s trace: https://jaeger.dev.storj.io/trace/7e44fa01de290116 FAIL (2026-04-06 11:26:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== loading segments... Apr 6 10:39:12.293175658 (743h39m9.88994668s) [upload=832686]: Got next segment. Inline: false Apr 6 10:39:12.293210466 (743h39m9.889981489s) [upload=832686]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 10:39:12.293215002 (743h39m9.889986025s) [upload=832686]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 10:39:12.319174409 (743h39m9.915945433s) [seg_pos={0 0}, upload=832686]: Begin upload segment... Apr 6 10:39:12.319264769 (743h39m9.916035785s) [seg_pos={0 0}, upload=832686]: Done begin upload segment. Apr 6 10:39:12.319310748 (743h39m9.91608178s) [noise=true , address=45.91.116.13:4166 , piece_id=CNVYP6YUT7E3VUCCSUR2ZWCDYU6QMOZL5GPZAMPQ5P2BVSD6HUDQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.319344730 (743h39m9.916115762s) [noise=true , address=45.91.116.6:4987 , piece_id=N2OUZHWZMNVIMAPTOHLIIT6NK5N3PVSHS6OX4ZGYSRUJIS5X3CDQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.319839277 (743h39m9.916610309s) [noise=true , address=193.149.164.67:4008 , piece_id=SJM4PFZHQJW7I5C3XWXIIAJF7DBXPQQYGHTB22AB7JKMG6VN7QZA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320290481 (743h39m9.917061503s) [noise=true , address=167.17.72.69:4734 , piece_id=I7HV3YRWA4IH7VE2I7SQFA42VHSYJVZCKA2KP4XQ4KSRJ6S7KPLA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320450536 (743h39m9.917221562s) [noise=true , address=45.91.116.9:4270 , piece_id=UZYXLU235ANTBCMIIQDUHM6LZUKAS64FUBJIY2NXGYTIAO776KFQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320733701 (743h39m9.917504727s) [noise=true , address=193.149.164.143:4694 , piece_id=GG3FQN4YC5QQVGFCFYWTZWOZPFQO5NL2XE4FJCSUKUCTY4L5EWMA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320988084 (743h39m9.917759097s) [noise=true , address=45.91.116.12:4257 , piece_id=GW7MFYT77HZYDY263WMUIABDAELV37MBKMBS57FFSFXCIUJ5JN7A, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321148099 (743h39m9.917919124s) [noise=true , address=145.79.227.37:4582 , piece_id=QFSMYYEUDOO5JURWQHBLBRLCOXYSP4M2OR4OBWKIZ5LCCXPXEJCQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321494813 (743h39m9.918265836s) [noise=true , address=45.91.116.11:4142 , piece_id=YAVLJVKDZWHWPN572Y5TX55ONWE2IDXVA6ABUF6KD7YRWCWR3YSQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321531682 (743h39m9.918302714s) [noise=true , address=193.149.164.68:4055 , piece_id=ARLLKHKO3AQJG52QH6EWHE55IKPQIEYCIIOR4RMIMU55TGYXYGKQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321879088 (743h39m9.918650182s) [noise=true , address=193.149.164.69:4104 , piece_id=Q3VXWW73OASMNUYH77EZBCQU5GAGPG3X3TKRLED43F3R6MZYZ2DQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322027388 (743h39m9.918798417s) [noise=true , address=193.149.164.66:4985 , piece_id=A6YQDCH7PTS6CP3IFWYMQ3HJGKFX6OSYQBATKUAIOXELHAEE5WBA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322241446 (743h39m9.919012475s) [noise=true , address=145.79.227.33:4878 , piece_id=YAEEULYYAHZP3RM6YAE55LC7S7DQLVV6JWIGYCY64GF6ITHJRQSQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322546472 (743h39m9.919317502s) [noise=true , address=167.17.72.70:4761 , piece_id=WFSQTX3PFW77TZQCSIUSHSD3LH74XTFKLYUMDNYG7MFERLUCSZHA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322583845 (743h39m9.919354868s) [noise=true , address=145.79.227.41:4116 , piece_id=E43SWVMVKWDULRK4YV6SDVJBCHWSAHSFRNACXSZZZQM2IU5JJ4KA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322937532 (743h39m9.919708565s) [noise=true , address=193.149.164.70:4105 , piece_id=7CKN7FHVF6OXY6ZPFZPM4DYNDOJNHGGF3PF3VFM7V2FSTAURFGWA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.323051576 (743h39m9.919822591s) [noise=true , address=77.90.31.17:4775 , piece_id=CFR7LFL2JZCVX4N7X2QQPJVJLZDSD6NZCNLNLSUJKU4DIZ7BBL3Q, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.323474343 (743h39m9.920245373s) [noise=true , address=167.17.72.68:4665 , piece_id=H7Y37VGEKXJGRPEZ2G4NG6WXTNNJ6YQMQI4NRUZYJTWMWIVF246Q, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.386273803 (743h39m9.983044815s) [upload=832686]: Next returned nil segment Apr 6 10:39:12.386288014 (743h39m9.983059037s) [upload=832686]: Waiting for error group managing segments... Apr 6 10:39:12.392899406 (743h39m9.989670418s) [noise=true , address=77.90.31.17:4775 , piece_id=CFR7LFL2JZCVX4N7X2QQPJVJLZDSD6NZCNLNLSUJKU4DIZ7BBL3Q, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.393163291 (743h39m9.989934315s) [noise=true , address=145.79.227.41:4116 , piece_id=E43SWVMVKWDULRK4YV6SDVJBCHWSAHSFRNACXSZZZQM2IU5JJ4KA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.667307989 (743h39m10.264079022s) [noise=true , address=167.17.72.68:4665 , piece_id=H7Y37VGEKXJGRPEZ2G4NG6WXTNNJ6YQMQI4NRUZYJTWMWIVF246Q, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.714497903 (743h39m10.311268937s) [noise=true , address=167.17.72.69:4734 , piece_id=I7HV3YRWA4IH7VE2I7SQFA42VHSYJVZCKA2KP4XQ4KSRJ6S7KPLA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.734199134 (743h39m10.330970147s) [noise=true , address=167.17.72.70:4761 , piece_id=WFSQTX3PFW77TZQCSIUSHSD3LH74XTFKLYUMDNYG7MFERLUCSZHA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.896708412 (743h39m10.493479439s) [noise=true , address=193.149.164.69:4104 , piece_id=Q3VXWW73OASMNUYH77EZBCQU5GAGPG3X3TKRLED43F3R6MZYZ2DQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.899988929 (743h39m10.496759964s) [noise=true , address=193.149.164.67:4008 , piece_id=SJM4PFZHQJW7I5C3XWXIIAJF7DBXPQQYGHTB22AB7JKMG6VN7QZA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.908433760 (743h39m10.505204783s) [noise=true , address=193.149.164.70:4105 , piece_id=7CKN7FHVF6OXY6ZPFZPM4DYNDOJNHGGF3PF3VFM7V2FSTAURFGWA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.916130090 (743h39m10.512901122s) [noise=true , address=193.149.164.68:4055 , piece_id=ARLLKHKO3AQJG52QH6EWHE55IKPQIEYCIIOR4RMIMU55TGYXYGKQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.956247905 (743h39m10.553018928s) [noise=true , address=193.149.164.66:4985 , piece_id=A6YQDCH7PTS6CP3IFWYMQ3HJGKFX6OSYQBATKUAIOXELHAEE5WBA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:13.000961275 (743h39m10.597732326s) [noise=true , address=193.149.164.143:4694 , piece_id=GG3FQN4YC5QQVGFCFYWTZWOZPFQO5NL2XE4FJCSUKUCTY4L5EWMA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:13.304880608 (743h39m10.90165162s) [noise=true , address=145.79.227.33:4878 , piece_id=YAEEULYYAHZP3RM6YAE55LC7S7DQLVV6JWIGYCY64GF6ITHJRQSQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:13.307202541 (743h39m10.903973568s) [noise=true , address=145.79.227.37:4582 , piece_id=QFSMYYEUDOO5JURWQHBLBRLCOXYSP4M2OR4OBWKIZ5LCCXPXEJCQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:40:12.257884464 (743h40m9.854655494s) [noise=true , address=45.91.116.12:4257 , piece_id=GW7MFYT77HZYDY263WMUIABDAELV37MBKMBS57FFSFXCIUJ5JN7A, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:12ZQ5a1TwxdeuJ6UwsVm9VsWnpZEBKC6N5yYUjyNTBkriSjvYvE, address:45.91.116.12:4257): protocol: rpc: dial tcp 45.91.116.12:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258122811 (743h40m9.854893836s) [noise=true , address=45.91.116.9:4270 , piece_id=UZYXLU235ANTBCMIIQDUHM6LZUKAS64FUBJIY2NXGYTIAO776KFQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:1Vr5WmL6Gem29LQTsCwQNFuqw6j4b1nj35AgbUWDA2Lyxf43YL, address:45.91.116.9:4270): protocol: read tcp 159.223.143.39:53356->45.91.116.9:4270: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258358693 (743h40m9.855129725s) [noise=true , address=45.91.116.6:4987 , piece_id=N2OUZHWZMNVIMAPTOHLIIT6NK5N3PVSHS6OX4ZGYSRUJIS5X3CDQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:12vPKcqLhopcpFxZ1LXwD8JXPLJB8a6HKfGnoyXsvKaL2oWUNao, address:45.91.116.6:4987): protocol: context deadline exceeded; context deadline exceeded Apr 6 10:40:12.258544102 (743h40m9.855315131s) [noise=true , address=45.91.116.11:4142 , piece_id=YAVLJVKDZWHWPN572Y5TX55ONWE2IDXVA6ABUF6KD7YRWCWR3YSQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:135mYyE4Gy1gcCXSocnb41e9u4z4MpqMjodmaRnrNs9AGS1YcX, address:45.91.116.11:4142): protocol: write tcp 159.223.143.39:34030->45.91.116.11:4142: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258881055 (743h40m9.855652086s) [noise=true , address=45.91.116.13:4166 , piece_id=CNVYP6YUT7E3VUCCSUR2ZWCDYU6QMOZL5GPZAMPQ5P2BVSD6HUDQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:12ajyozgSuf68kn8sUuEmuSfYCcxzU8Cwmg1EXVt9WV6dKSopy7, address:45.91.116.13:4166): protocol: write tcp 159.223.143.39:45960->45.91.116.13:4166: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258934014 (743h40m9.855705039s) [seg_pos={0 0}, upload=832686]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 10:40:12.278983295 (743h40m9.875754318s) [upload=832686]: Done uploading segments... Apr 6 10:40:12.279076909 (743h40m9.875847939s) [upload=832686]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.334741848s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [X...........................................................] duration: 692.18316ms trace: https://jaeger.dev.storj.io/trace/4736d76b51c361a3 FAIL (2026-04-06 11:26:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 692.213691ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 687.451756ms trace: https://jaeger.dev.storj.io/trace/7756830e434d8498 FAIL (2026-04-06 11:26:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 687.486823ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 1.211961315s trace: https://jaeger.dev.storj.io/trace/7e44fa01de290116 FAIL (2026-04-06 11:25:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== loading segments... Apr 6 10:39:12.293175658 (743h39m9.88994668s) [upload=832686]: Got next segment. Inline: false Apr 6 10:39:12.293210466 (743h39m9.889981489s) [upload=832686]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 10:39:12.293215002 (743h39m9.889986025s) [upload=832686]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 10:39:12.319174409 (743h39m9.915945433s) [seg_pos={0 0}, upload=832686]: Begin upload segment... Apr 6 10:39:12.319264769 (743h39m9.916035785s) [seg_pos={0 0}, upload=832686]: Done begin upload segment. Apr 6 10:39:12.319310748 (743h39m9.91608178s) [noise=true , address=45.91.116.13:4166 , piece_id=CNVYP6YUT7E3VUCCSUR2ZWCDYU6QMOZL5GPZAMPQ5P2BVSD6HUDQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.319344730 (743h39m9.916115762s) [noise=true , address=45.91.116.6:4987 , piece_id=N2OUZHWZMNVIMAPTOHLIIT6NK5N3PVSHS6OX4ZGYSRUJIS5X3CDQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.319839277 (743h39m9.916610309s) [noise=true , address=193.149.164.67:4008 , piece_id=SJM4PFZHQJW7I5C3XWXIIAJF7DBXPQQYGHTB22AB7JKMG6VN7QZA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320290481 (743h39m9.917061503s) [noise=true , address=167.17.72.69:4734 , piece_id=I7HV3YRWA4IH7VE2I7SQFA42VHSYJVZCKA2KP4XQ4KSRJ6S7KPLA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320450536 (743h39m9.917221562s) [noise=true , address=45.91.116.9:4270 , piece_id=UZYXLU235ANTBCMIIQDUHM6LZUKAS64FUBJIY2NXGYTIAO776KFQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320733701 (743h39m9.917504727s) [noise=true , address=193.149.164.143:4694 , piece_id=GG3FQN4YC5QQVGFCFYWTZWOZPFQO5NL2XE4FJCSUKUCTY4L5EWMA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320988084 (743h39m9.917759097s) [noise=true , address=45.91.116.12:4257 , piece_id=GW7MFYT77HZYDY263WMUIABDAELV37MBKMBS57FFSFXCIUJ5JN7A, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321148099 (743h39m9.917919124s) [noise=true , address=145.79.227.37:4582 , piece_id=QFSMYYEUDOO5JURWQHBLBRLCOXYSP4M2OR4OBWKIZ5LCCXPXEJCQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321494813 (743h39m9.918265836s) [noise=true , address=45.91.116.11:4142 , piece_id=YAVLJVKDZWHWPN572Y5TX55ONWE2IDXVA6ABUF6KD7YRWCWR3YSQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321531682 (743h39m9.918302714s) [noise=true , address=193.149.164.68:4055 , piece_id=ARLLKHKO3AQJG52QH6EWHE55IKPQIEYCIIOR4RMIMU55TGYXYGKQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321879088 (743h39m9.918650182s) [noise=true , address=193.149.164.69:4104 , piece_id=Q3VXWW73OASMNUYH77EZBCQU5GAGPG3X3TKRLED43F3R6MZYZ2DQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322027388 (743h39m9.918798417s) [noise=true , address=193.149.164.66:4985 , piece_id=A6YQDCH7PTS6CP3IFWYMQ3HJGKFX6OSYQBATKUAIOXELHAEE5WBA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322241446 (743h39m9.919012475s) [noise=true , address=145.79.227.33:4878 , piece_id=YAEEULYYAHZP3RM6YAE55LC7S7DQLVV6JWIGYCY64GF6ITHJRQSQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322546472 (743h39m9.919317502s) [noise=true , address=167.17.72.70:4761 , piece_id=WFSQTX3PFW77TZQCSIUSHSD3LH74XTFKLYUMDNYG7MFERLUCSZHA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322583845 (743h39m9.919354868s) [noise=true , address=145.79.227.41:4116 , piece_id=E43SWVMVKWDULRK4YV6SDVJBCHWSAHSFRNACXSZZZQM2IU5JJ4KA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322937532 (743h39m9.919708565s) [noise=true , address=193.149.164.70:4105 , piece_id=7CKN7FHVF6OXY6ZPFZPM4DYNDOJNHGGF3PF3VFM7V2FSTAURFGWA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.323051576 (743h39m9.919822591s) [noise=true , address=77.90.31.17:4775 , piece_id=CFR7LFL2JZCVX4N7X2QQPJVJLZDSD6NZCNLNLSUJKU4DIZ7BBL3Q, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.323474343 (743h39m9.920245373s) [noise=true , address=167.17.72.68:4665 , piece_id=H7Y37VGEKXJGRPEZ2G4NG6WXTNNJ6YQMQI4NRUZYJTWMWIVF246Q, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.386273803 (743h39m9.983044815s) [upload=832686]: Next returned nil segment Apr 6 10:39:12.386288014 (743h39m9.983059037s) [upload=832686]: Waiting for error group managing segments... Apr 6 10:39:12.392899406 (743h39m9.989670418s) [noise=true , address=77.90.31.17:4775 , piece_id=CFR7LFL2JZCVX4N7X2QQPJVJLZDSD6NZCNLNLSUJKU4DIZ7BBL3Q, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.393163291 (743h39m9.989934315s) [noise=true , address=145.79.227.41:4116 , piece_id=E43SWVMVKWDULRK4YV6SDVJBCHWSAHSFRNACXSZZZQM2IU5JJ4KA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.667307989 (743h39m10.264079022s) [noise=true , address=167.17.72.68:4665 , piece_id=H7Y37VGEKXJGRPEZ2G4NG6WXTNNJ6YQMQI4NRUZYJTWMWIVF246Q, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.714497903 (743h39m10.311268937s) [noise=true , address=167.17.72.69:4734 , piece_id=I7HV3YRWA4IH7VE2I7SQFA42VHSYJVZCKA2KP4XQ4KSRJ6S7KPLA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.734199134 (743h39m10.330970147s) [noise=true , address=167.17.72.70:4761 , piece_id=WFSQTX3PFW77TZQCSIUSHSD3LH74XTFKLYUMDNYG7MFERLUCSZHA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.896708412 (743h39m10.493479439s) [noise=true , address=193.149.164.69:4104 , piece_id=Q3VXWW73OASMNUYH77EZBCQU5GAGPG3X3TKRLED43F3R6MZYZ2DQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.899988929 (743h39m10.496759964s) [noise=true , address=193.149.164.67:4008 , piece_id=SJM4PFZHQJW7I5C3XWXIIAJF7DBXPQQYGHTB22AB7JKMG6VN7QZA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.908433760 (743h39m10.505204783s) [noise=true , address=193.149.164.70:4105 , piece_id=7CKN7FHVF6OXY6ZPFZPM4DYNDOJNHGGF3PF3VFM7V2FSTAURFGWA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.916130090 (743h39m10.512901122s) [noise=true , address=193.149.164.68:4055 , piece_id=ARLLKHKO3AQJG52QH6EWHE55IKPQIEYCIIOR4RMIMU55TGYXYGKQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.956247905 (743h39m10.553018928s) [noise=true , address=193.149.164.66:4985 , piece_id=A6YQDCH7PTS6CP3IFWYMQ3HJGKFX6OSYQBATKUAIOXELHAEE5WBA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:13.000961275 (743h39m10.597732326s) [noise=true , address=193.149.164.143:4694 , piece_id=GG3FQN4YC5QQVGFCFYWTZWOZPFQO5NL2XE4FJCSUKUCTY4L5EWMA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:13.304880608 (743h39m10.90165162s) [noise=true , address=145.79.227.33:4878 , piece_id=YAEEULYYAHZP3RM6YAE55LC7S7DQLVV6JWIGYCY64GF6ITHJRQSQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:13.307202541 (743h39m10.903973568s) [noise=true , address=145.79.227.37:4582 , piece_id=QFSMYYEUDOO5JURWQHBLBRLCOXYSP4M2OR4OBWKIZ5LCCXPXEJCQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:40:12.257884464 (743h40m9.854655494s) [noise=true , address=45.91.116.12:4257 , piece_id=GW7MFYT77HZYDY263WMUIABDAELV37MBKMBS57FFSFXCIUJ5JN7A, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:12ZQ5a1TwxdeuJ6UwsVm9VsWnpZEBKC6N5yYUjyNTBkriSjvYvE, address:45.91.116.12:4257): protocol: rpc: dial tcp 45.91.116.12:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258122811 (743h40m9.854893836s) [noise=true , address=45.91.116.9:4270 , piece_id=UZYXLU235ANTBCMIIQDUHM6LZUKAS64FUBJIY2NXGYTIAO776KFQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:1Vr5WmL6Gem29LQTsCwQNFuqw6j4b1nj35AgbUWDA2Lyxf43YL, address:45.91.116.9:4270): protocol: read tcp 159.223.143.39:53356->45.91.116.9:4270: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258358693 (743h40m9.855129725s) [noise=true , address=45.91.116.6:4987 , piece_id=N2OUZHWZMNVIMAPTOHLIIT6NK5N3PVSHS6OX4ZGYSRUJIS5X3CDQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:12vPKcqLhopcpFxZ1LXwD8JXPLJB8a6HKfGnoyXsvKaL2oWUNao, address:45.91.116.6:4987): protocol: context deadline exceeded; context deadline exceeded Apr 6 10:40:12.258544102 (743h40m9.855315131s) [noise=true , address=45.91.116.11:4142 , piece_id=YAVLJVKDZWHWPN572Y5TX55ONWE2IDXVA6ABUF6KD7YRWCWR3YSQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:135mYyE4Gy1gcCXSocnb41e9u4z4MpqMjodmaRnrNs9AGS1YcX, address:45.91.116.11:4142): protocol: write tcp 159.223.143.39:34030->45.91.116.11:4142: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258881055 (743h40m9.855652086s) [noise=true , address=45.91.116.13:4166 , piece_id=CNVYP6YUT7E3VUCCSUR2ZWCDYU6QMOZL5GPZAMPQ5P2BVSD6HUDQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:12ajyozgSuf68kn8sUuEmuSfYCcxzU8Cwmg1EXVt9WV6dKSopy7, address:45.91.116.13:4166): protocol: write tcp 159.223.143.39:45960->45.91.116.13:4166: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258934014 (743h40m9.855705039s) [seg_pos={0 0}, upload=832686]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 10:40:12.278983295 (743h40m9.875754318s) [upload=832686]: Done uploading segments... Apr 6 10:40:12.279076909 (743h40m9.875847939s) [upload=832686]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.21198907s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 1.701588481s trace: https://jaeger.dev.storj.io/trace/7e44fa01de290116 FAIL (2026-04-06 11:25:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== loading segments... Apr 6 10:39:12.293175658 (743h39m9.88994668s) [upload=832686]: Got next segment. Inline: false Apr 6 10:39:12.293210466 (743h39m9.889981489s) [upload=832686]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 10:39:12.293215002 (743h39m9.889986025s) [upload=832686]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 10:39:12.319174409 (743h39m9.915945433s) [seg_pos={0 0}, upload=832686]: Begin upload segment... Apr 6 10:39:12.319264769 (743h39m9.916035785s) [seg_pos={0 0}, upload=832686]: Done begin upload segment. Apr 6 10:39:12.319310748 (743h39m9.91608178s) [noise=true , address=45.91.116.13:4166 , piece_id=CNVYP6YUT7E3VUCCSUR2ZWCDYU6QMOZL5GPZAMPQ5P2BVSD6HUDQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.319344730 (743h39m9.916115762s) [noise=true , address=45.91.116.6:4987 , piece_id=N2OUZHWZMNVIMAPTOHLIIT6NK5N3PVSHS6OX4ZGYSRUJIS5X3CDQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.319839277 (743h39m9.916610309s) [noise=true , address=193.149.164.67:4008 , piece_id=SJM4PFZHQJW7I5C3XWXIIAJF7DBXPQQYGHTB22AB7JKMG6VN7QZA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320290481 (743h39m9.917061503s) [noise=true , address=167.17.72.69:4734 , piece_id=I7HV3YRWA4IH7VE2I7SQFA42VHSYJVZCKA2KP4XQ4KSRJ6S7KPLA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320450536 (743h39m9.917221562s) [noise=true , address=45.91.116.9:4270 , piece_id=UZYXLU235ANTBCMIIQDUHM6LZUKAS64FUBJIY2NXGYTIAO776KFQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320733701 (743h39m9.917504727s) [noise=true , address=193.149.164.143:4694 , piece_id=GG3FQN4YC5QQVGFCFYWTZWOZPFQO5NL2XE4FJCSUKUCTY4L5EWMA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320988084 (743h39m9.917759097s) [noise=true , address=45.91.116.12:4257 , piece_id=GW7MFYT77HZYDY263WMUIABDAELV37MBKMBS57FFSFXCIUJ5JN7A, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321148099 (743h39m9.917919124s) [noise=true , address=145.79.227.37:4582 , piece_id=QFSMYYEUDOO5JURWQHBLBRLCOXYSP4M2OR4OBWKIZ5LCCXPXEJCQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321494813 (743h39m9.918265836s) [noise=true , address=45.91.116.11:4142 , piece_id=YAVLJVKDZWHWPN572Y5TX55ONWE2IDXVA6ABUF6KD7YRWCWR3YSQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321531682 (743h39m9.918302714s) [noise=true , address=193.149.164.68:4055 , piece_id=ARLLKHKO3AQJG52QH6EWHE55IKPQIEYCIIOR4RMIMU55TGYXYGKQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321879088 (743h39m9.918650182s) [noise=true , address=193.149.164.69:4104 , piece_id=Q3VXWW73OASMNUYH77EZBCQU5GAGPG3X3TKRLED43F3R6MZYZ2DQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322027388 (743h39m9.918798417s) [noise=true , address=193.149.164.66:4985 , piece_id=A6YQDCH7PTS6CP3IFWYMQ3HJGKFX6OSYQBATKUAIOXELHAEE5WBA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322241446 (743h39m9.919012475s) [noise=true , address=145.79.227.33:4878 , piece_id=YAEEULYYAHZP3RM6YAE55LC7S7DQLVV6JWIGYCY64GF6ITHJRQSQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322546472 (743h39m9.919317502s) [noise=true , address=167.17.72.70:4761 , piece_id=WFSQTX3PFW77TZQCSIUSHSD3LH74XTFKLYUMDNYG7MFERLUCSZHA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322583845 (743h39m9.919354868s) [noise=true , address=145.79.227.41:4116 , piece_id=E43SWVMVKWDULRK4YV6SDVJBCHWSAHSFRNACXSZZZQM2IU5JJ4KA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322937532 (743h39m9.919708565s) [noise=true , address=193.149.164.70:4105 , piece_id=7CKN7FHVF6OXY6ZPFZPM4DYNDOJNHGGF3PF3VFM7V2FSTAURFGWA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.323051576 (743h39m9.919822591s) [noise=true , address=77.90.31.17:4775 , piece_id=CFR7LFL2JZCVX4N7X2QQPJVJLZDSD6NZCNLNLSUJKU4DIZ7BBL3Q, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.323474343 (743h39m9.920245373s) [noise=true , address=167.17.72.68:4665 , piece_id=H7Y37VGEKXJGRPEZ2G4NG6WXTNNJ6YQMQI4NRUZYJTWMWIVF246Q, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.386273803 (743h39m9.983044815s) [upload=832686]: Next returned nil segment Apr 6 10:39:12.386288014 (743h39m9.983059037s) [upload=832686]: Waiting for error group managing segments... Apr 6 10:39:12.392899406 (743h39m9.989670418s) [noise=true , address=77.90.31.17:4775 , piece_id=CFR7LFL2JZCVX4N7X2QQPJVJLZDSD6NZCNLNLSUJKU4DIZ7BBL3Q, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.393163291 (743h39m9.989934315s) [noise=true , address=145.79.227.41:4116 , piece_id=E43SWVMVKWDULRK4YV6SDVJBCHWSAHSFRNACXSZZZQM2IU5JJ4KA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.667307989 (743h39m10.264079022s) [noise=true , address=167.17.72.68:4665 , piece_id=H7Y37VGEKXJGRPEZ2G4NG6WXTNNJ6YQMQI4NRUZYJTWMWIVF246Q, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.714497903 (743h39m10.311268937s) [noise=true , address=167.17.72.69:4734 , piece_id=I7HV3YRWA4IH7VE2I7SQFA42VHSYJVZCKA2KP4XQ4KSRJ6S7KPLA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.734199134 (743h39m10.330970147s) [noise=true , address=167.17.72.70:4761 , piece_id=WFSQTX3PFW77TZQCSIUSHSD3LH74XTFKLYUMDNYG7MFERLUCSZHA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.896708412 (743h39m10.493479439s) [noise=true , address=193.149.164.69:4104 , piece_id=Q3VXWW73OASMNUYH77EZBCQU5GAGPG3X3TKRLED43F3R6MZYZ2DQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.899988929 (743h39m10.496759964s) [noise=true , address=193.149.164.67:4008 , piece_id=SJM4PFZHQJW7I5C3XWXIIAJF7DBXPQQYGHTB22AB7JKMG6VN7QZA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.908433760 (743h39m10.505204783s) [noise=true , address=193.149.164.70:4105 , piece_id=7CKN7FHVF6OXY6ZPFZPM4DYNDOJNHGGF3PF3VFM7V2FSTAURFGWA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.916130090 (743h39m10.512901122s) [noise=true , address=193.149.164.68:4055 , piece_id=ARLLKHKO3AQJG52QH6EWHE55IKPQIEYCIIOR4RMIMU55TGYXYGKQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.956247905 (743h39m10.553018928s) [noise=true , address=193.149.164.66:4985 , piece_id=A6YQDCH7PTS6CP3IFWYMQ3HJGKFX6OSYQBATKUAIOXELHAEE5WBA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:13.000961275 (743h39m10.597732326s) [noise=true , address=193.149.164.143:4694 , piece_id=GG3FQN4YC5QQVGFCFYWTZWOZPFQO5NL2XE4FJCSUKUCTY4L5EWMA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:13.304880608 (743h39m10.90165162s) [noise=true , address=145.79.227.33:4878 , piece_id=YAEEULYYAHZP3RM6YAE55LC7S7DQLVV6JWIGYCY64GF6ITHJRQSQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:13.307202541 (743h39m10.903973568s) [noise=true , address=145.79.227.37:4582 , piece_id=QFSMYYEUDOO5JURWQHBLBRLCOXYSP4M2OR4OBWKIZ5LCCXPXEJCQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:40:12.257884464 (743h40m9.854655494s) [noise=true , address=45.91.116.12:4257 , piece_id=GW7MFYT77HZYDY263WMUIABDAELV37MBKMBS57FFSFXCIUJ5JN7A, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:12ZQ5a1TwxdeuJ6UwsVm9VsWnpZEBKC6N5yYUjyNTBkriSjvYvE, address:45.91.116.12:4257): protocol: rpc: dial tcp 45.91.116.12:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258122811 (743h40m9.854893836s) [noise=true , address=45.91.116.9:4270 , piece_id=UZYXLU235ANTBCMIIQDUHM6LZUKAS64FUBJIY2NXGYTIAO776KFQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:1Vr5WmL6Gem29LQTsCwQNFuqw6j4b1nj35AgbUWDA2Lyxf43YL, address:45.91.116.9:4270): protocol: read tcp 159.223.143.39:53356->45.91.116.9:4270: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258358693 (743h40m9.855129725s) [noise=true , address=45.91.116.6:4987 , piece_id=N2OUZHWZMNVIMAPTOHLIIT6NK5N3PVSHS6OX4ZGYSRUJIS5X3CDQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:12vPKcqLhopcpFxZ1LXwD8JXPLJB8a6HKfGnoyXsvKaL2oWUNao, address:45.91.116.6:4987): protocol: context deadline exceeded; context deadline exceeded Apr 6 10:40:12.258544102 (743h40m9.855315131s) [noise=true , address=45.91.116.11:4142 , piece_id=YAVLJVKDZWHWPN572Y5TX55ONWE2IDXVA6ABUF6KD7YRWCWR3YSQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:135mYyE4Gy1gcCXSocnb41e9u4z4MpqMjodmaRnrNs9AGS1YcX, address:45.91.116.11:4142): protocol: write tcp 159.223.143.39:34030->45.91.116.11:4142: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258881055 (743h40m9.855652086s) [noise=true , address=45.91.116.13:4166 , piece_id=CNVYP6YUT7E3VUCCSUR2ZWCDYU6QMOZL5GPZAMPQ5P2BVSD6HUDQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:12ajyozgSuf68kn8sUuEmuSfYCcxzU8Cwmg1EXVt9WV6dKSopy7, address:45.91.116.13:4166): protocol: write tcp 159.223.143.39:45960->45.91.116.13:4166: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258934014 (743h40m9.855705039s) [seg_pos={0 0}, upload=832686]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 10:40:12.278983295 (743h40m9.875754318s) [upload=832686]: Done uploading segments... Apr 6 10:40:12.279076909 (743h40m9.875847939s) [upload=832686]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.701619625s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [XX..........................................................] duration: 505.453878ms trace: https://jaeger.dev.storj.io/trace/4736d76b51c361a3 FAIL (2026-04-06 11:25:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 505.506272ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 1.029055125s trace: https://jaeger.dev.storj.io/trace/7756830e434d8498 FAIL (2026-04-06 11:24:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.029081534s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 1.250590477s trace: https://jaeger.dev.storj.io/trace/7e44fa01de290116 FAIL (2026-04-06 11:24:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== loading segments... Apr 6 10:39:12.293175658 (743h39m9.88994668s) [upload=832686]: Got next segment. Inline: false Apr 6 10:39:12.293210466 (743h39m9.889981489s) [upload=832686]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 10:39:12.293215002 (743h39m9.889986025s) [upload=832686]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 10:39:12.319174409 (743h39m9.915945433s) [seg_pos={0 0}, upload=832686]: Begin upload segment... Apr 6 10:39:12.319264769 (743h39m9.916035785s) [seg_pos={0 0}, upload=832686]: Done begin upload segment. Apr 6 10:39:12.319310748 (743h39m9.91608178s) [noise=true , address=45.91.116.13:4166 , piece_id=CNVYP6YUT7E3VUCCSUR2ZWCDYU6QMOZL5GPZAMPQ5P2BVSD6HUDQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.319344730 (743h39m9.916115762s) [noise=true , address=45.91.116.6:4987 , piece_id=N2OUZHWZMNVIMAPTOHLIIT6NK5N3PVSHS6OX4ZGYSRUJIS5X3CDQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.319839277 (743h39m9.916610309s) [noise=true , address=193.149.164.67:4008 , piece_id=SJM4PFZHQJW7I5C3XWXIIAJF7DBXPQQYGHTB22AB7JKMG6VN7QZA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320290481 (743h39m9.917061503s) [noise=true , address=167.17.72.69:4734 , piece_id=I7HV3YRWA4IH7VE2I7SQFA42VHSYJVZCKA2KP4XQ4KSRJ6S7KPLA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320450536 (743h39m9.917221562s) [noise=true , address=45.91.116.9:4270 , piece_id=UZYXLU235ANTBCMIIQDUHM6LZUKAS64FUBJIY2NXGYTIAO776KFQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320733701 (743h39m9.917504727s) [noise=true , address=193.149.164.143:4694 , piece_id=GG3FQN4YC5QQVGFCFYWTZWOZPFQO5NL2XE4FJCSUKUCTY4L5EWMA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320988084 (743h39m9.917759097s) [noise=true , address=45.91.116.12:4257 , piece_id=GW7MFYT77HZYDY263WMUIABDAELV37MBKMBS57FFSFXCIUJ5JN7A, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321148099 (743h39m9.917919124s) [noise=true , address=145.79.227.37:4582 , piece_id=QFSMYYEUDOO5JURWQHBLBRLCOXYSP4M2OR4OBWKIZ5LCCXPXEJCQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321494813 (743h39m9.918265836s) [noise=true , address=45.91.116.11:4142 , piece_id=YAVLJVKDZWHWPN572Y5TX55ONWE2IDXVA6ABUF6KD7YRWCWR3YSQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321531682 (743h39m9.918302714s) [noise=true , address=193.149.164.68:4055 , piece_id=ARLLKHKO3AQJG52QH6EWHE55IKPQIEYCIIOR4RMIMU55TGYXYGKQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321879088 (743h39m9.918650182s) [noise=true , address=193.149.164.69:4104 , piece_id=Q3VXWW73OASMNUYH77EZBCQU5GAGPG3X3TKRLED43F3R6MZYZ2DQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322027388 (743h39m9.918798417s) [noise=true , address=193.149.164.66:4985 , piece_id=A6YQDCH7PTS6CP3IFWYMQ3HJGKFX6OSYQBATKUAIOXELHAEE5WBA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322241446 (743h39m9.919012475s) [noise=true , address=145.79.227.33:4878 , piece_id=YAEEULYYAHZP3RM6YAE55LC7S7DQLVV6JWIGYCY64GF6ITHJRQSQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322546472 (743h39m9.919317502s) [noise=true , address=167.17.72.70:4761 , piece_id=WFSQTX3PFW77TZQCSIUSHSD3LH74XTFKLYUMDNYG7MFERLUCSZHA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322583845 (743h39m9.919354868s) [noise=true , address=145.79.227.41:4116 , piece_id=E43SWVMVKWDULRK4YV6SDVJBCHWSAHSFRNACXSZZZQM2IU5JJ4KA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322937532 (743h39m9.919708565s) [noise=true , address=193.149.164.70:4105 , piece_id=7CKN7FHVF6OXY6ZPFZPM4DYNDOJNHGGF3PF3VFM7V2FSTAURFGWA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.323051576 (743h39m9.919822591s) [noise=true , address=77.90.31.17:4775 , piece_id=CFR7LFL2JZCVX4N7X2QQPJVJLZDSD6NZCNLNLSUJKU4DIZ7BBL3Q, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.323474343 (743h39m9.920245373s) [noise=true , address=167.17.72.68:4665 , piece_id=H7Y37VGEKXJGRPEZ2G4NG6WXTNNJ6YQMQI4NRUZYJTWMWIVF246Q, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.386273803 (743h39m9.983044815s) [upload=832686]: Next returned nil segment Apr 6 10:39:12.386288014 (743h39m9.983059037s) [upload=832686]: Waiting for error group managing segments... Apr 6 10:39:12.392899406 (743h39m9.989670418s) [noise=true , address=77.90.31.17:4775 , piece_id=CFR7LFL2JZCVX4N7X2QQPJVJLZDSD6NZCNLNLSUJKU4DIZ7BBL3Q, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.393163291 (743h39m9.989934315s) [noise=true , address=145.79.227.41:4116 , piece_id=E43SWVMVKWDULRK4YV6SDVJBCHWSAHSFRNACXSZZZQM2IU5JJ4KA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.667307989 (743h39m10.264079022s) [noise=true , address=167.17.72.68:4665 , piece_id=H7Y37VGEKXJGRPEZ2G4NG6WXTNNJ6YQMQI4NRUZYJTWMWIVF246Q, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.714497903 (743h39m10.311268937s) [noise=true , address=167.17.72.69:4734 , piece_id=I7HV3YRWA4IH7VE2I7SQFA42VHSYJVZCKA2KP4XQ4KSRJ6S7KPLA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.734199134 (743h39m10.330970147s) [noise=true , address=167.17.72.70:4761 , piece_id=WFSQTX3PFW77TZQCSIUSHSD3LH74XTFKLYUMDNYG7MFERLUCSZHA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.896708412 (743h39m10.493479439s) [noise=true , address=193.149.164.69:4104 , piece_id=Q3VXWW73OASMNUYH77EZBCQU5GAGPG3X3TKRLED43F3R6MZYZ2DQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.899988929 (743h39m10.496759964s) [noise=true , address=193.149.164.67:4008 , piece_id=SJM4PFZHQJW7I5C3XWXIIAJF7DBXPQQYGHTB22AB7JKMG6VN7QZA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.908433760 (743h39m10.505204783s) [noise=true , address=193.149.164.70:4105 , piece_id=7CKN7FHVF6OXY6ZPFZPM4DYNDOJNHGGF3PF3VFM7V2FSTAURFGWA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.916130090 (743h39m10.512901122s) [noise=true , address=193.149.164.68:4055 , piece_id=ARLLKHKO3AQJG52QH6EWHE55IKPQIEYCIIOR4RMIMU55TGYXYGKQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.956247905 (743h39m10.553018928s) [noise=true , address=193.149.164.66:4985 , piece_id=A6YQDCH7PTS6CP3IFWYMQ3HJGKFX6OSYQBATKUAIOXELHAEE5WBA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:13.000961275 (743h39m10.597732326s) [noise=true , address=193.149.164.143:4694 , piece_id=GG3FQN4YC5QQVGFCFYWTZWOZPFQO5NL2XE4FJCSUKUCTY4L5EWMA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:13.304880608 (743h39m10.90165162s) [noise=true , address=145.79.227.33:4878 , piece_id=YAEEULYYAHZP3RM6YAE55LC7S7DQLVV6JWIGYCY64GF6ITHJRQSQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:13.307202541 (743h39m10.903973568s) [noise=true , address=145.79.227.37:4582 , piece_id=QFSMYYEUDOO5JURWQHBLBRLCOXYSP4M2OR4OBWKIZ5LCCXPXEJCQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:40:12.257884464 (743h40m9.854655494s) [noise=true , address=45.91.116.12:4257 , piece_id=GW7MFYT77HZYDY263WMUIABDAELV37MBKMBS57FFSFXCIUJ5JN7A, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:12ZQ5a1TwxdeuJ6UwsVm9VsWnpZEBKC6N5yYUjyNTBkriSjvYvE, address:45.91.116.12:4257): protocol: rpc: dial tcp 45.91.116.12:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258122811 (743h40m9.854893836s) [noise=true , address=45.91.116.9:4270 , piece_id=UZYXLU235ANTBCMIIQDUHM6LZUKAS64FUBJIY2NXGYTIAO776KFQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:1Vr5WmL6Gem29LQTsCwQNFuqw6j4b1nj35AgbUWDA2Lyxf43YL, address:45.91.116.9:4270): protocol: read tcp 159.223.143.39:53356->45.91.116.9:4270: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258358693 (743h40m9.855129725s) [noise=true , address=45.91.116.6:4987 , piece_id=N2OUZHWZMNVIMAPTOHLIIT6NK5N3PVSHS6OX4ZGYSRUJIS5X3CDQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:12vPKcqLhopcpFxZ1LXwD8JXPLJB8a6HKfGnoyXsvKaL2oWUNao, address:45.91.116.6:4987): protocol: context deadline exceeded; context deadline exceeded Apr 6 10:40:12.258544102 (743h40m9.855315131s) [noise=true , address=45.91.116.11:4142 , piece_id=YAVLJVKDZWHWPN572Y5TX55ONWE2IDXVA6ABUF6KD7YRWCWR3YSQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:135mYyE4Gy1gcCXSocnb41e9u4z4MpqMjodmaRnrNs9AGS1YcX, address:45.91.116.11:4142): protocol: write tcp 159.223.143.39:34030->45.91.116.11:4142: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258881055 (743h40m9.855652086s) [noise=true , address=45.91.116.13:4166 , piece_id=CNVYP6YUT7E3VUCCSUR2ZWCDYU6QMOZL5GPZAMPQ5P2BVSD6HUDQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:12ajyozgSuf68kn8sUuEmuSfYCcxzU8Cwmg1EXVt9WV6dKSopy7, address:45.91.116.13:4166): protocol: write tcp 159.223.143.39:45960->45.91.116.13:4166: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258934014 (743h40m9.855705039s) [seg_pos={0 0}, upload=832686]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 10:40:12.278983295 (743h40m9.875754318s) [upload=832686]: Done uploading segments... Apr 6 10:40:12.279076909 (743h40m9.875847939s) [upload=832686]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.250620634s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [XXX.........................................................] duration: 454.486012ms trace: https://jaeger.dev.storj.io/trace/4736d76b51c361a3 FAIL (2026-04-06 11:23:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 454.750993ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 544.989096ms trace: https://jaeger.dev.storj.io/trace/7756830e434d8498 FAIL (2026-04-06 11:23:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 545.017728ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.XXX........................................................] duration: 428.281564ms trace: https://jaeger.dev.storj.io/trace/4736d76b51c361a3 FAIL (2026-04-06 11:23:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 428.330099ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 1.625831934s trace: https://jaeger.dev.storj.io/trace/7e44fa01de290116 FAIL (2026-04-06 11:23:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== loading segments... Apr 6 10:39:12.293175658 (743h39m9.88994668s) [upload=832686]: Got next segment. Inline: false Apr 6 10:39:12.293210466 (743h39m9.889981489s) [upload=832686]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 10:39:12.293215002 (743h39m9.889986025s) [upload=832686]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 10:39:12.319174409 (743h39m9.915945433s) [seg_pos={0 0}, upload=832686]: Begin upload segment... Apr 6 10:39:12.319264769 (743h39m9.916035785s) [seg_pos={0 0}, upload=832686]: Done begin upload segment. Apr 6 10:39:12.319310748 (743h39m9.91608178s) [noise=true , address=45.91.116.13:4166 , piece_id=CNVYP6YUT7E3VUCCSUR2ZWCDYU6QMOZL5GPZAMPQ5P2BVSD6HUDQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.319344730 (743h39m9.916115762s) [noise=true , address=45.91.116.6:4987 , piece_id=N2OUZHWZMNVIMAPTOHLIIT6NK5N3PVSHS6OX4ZGYSRUJIS5X3CDQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.319839277 (743h39m9.916610309s) [noise=true , address=193.149.164.67:4008 , piece_id=SJM4PFZHQJW7I5C3XWXIIAJF7DBXPQQYGHTB22AB7JKMG6VN7QZA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320290481 (743h39m9.917061503s) [noise=true , address=167.17.72.69:4734 , piece_id=I7HV3YRWA4IH7VE2I7SQFA42VHSYJVZCKA2KP4XQ4KSRJ6S7KPLA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320450536 (743h39m9.917221562s) [noise=true , address=45.91.116.9:4270 , piece_id=UZYXLU235ANTBCMIIQDUHM6LZUKAS64FUBJIY2NXGYTIAO776KFQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320733701 (743h39m9.917504727s) [noise=true , address=193.149.164.143:4694 , piece_id=GG3FQN4YC5QQVGFCFYWTZWOZPFQO5NL2XE4FJCSUKUCTY4L5EWMA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320988084 (743h39m9.917759097s) [noise=true , address=45.91.116.12:4257 , piece_id=GW7MFYT77HZYDY263WMUIABDAELV37MBKMBS57FFSFXCIUJ5JN7A, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321148099 (743h39m9.917919124s) [noise=true , address=145.79.227.37:4582 , piece_id=QFSMYYEUDOO5JURWQHBLBRLCOXYSP4M2OR4OBWKIZ5LCCXPXEJCQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321494813 (743h39m9.918265836s) [noise=true , address=45.91.116.11:4142 , piece_id=YAVLJVKDZWHWPN572Y5TX55ONWE2IDXVA6ABUF6KD7YRWCWR3YSQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321531682 (743h39m9.918302714s) [noise=true , address=193.149.164.68:4055 , piece_id=ARLLKHKO3AQJG52QH6EWHE55IKPQIEYCIIOR4RMIMU55TGYXYGKQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321879088 (743h39m9.918650182s) [noise=true , address=193.149.164.69:4104 , piece_id=Q3VXWW73OASMNUYH77EZBCQU5GAGPG3X3TKRLED43F3R6MZYZ2DQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322027388 (743h39m9.918798417s) [noise=true , address=193.149.164.66:4985 , piece_id=A6YQDCH7PTS6CP3IFWYMQ3HJGKFX6OSYQBATKUAIOXELHAEE5WBA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322241446 (743h39m9.919012475s) [noise=true , address=145.79.227.33:4878 , piece_id=YAEEULYYAHZP3RM6YAE55LC7S7DQLVV6JWIGYCY64GF6ITHJRQSQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322546472 (743h39m9.919317502s) [noise=true , address=167.17.72.70:4761 , piece_id=WFSQTX3PFW77TZQCSIUSHSD3LH74XTFKLYUMDNYG7MFERLUCSZHA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322583845 (743h39m9.919354868s) [noise=true , address=145.79.227.41:4116 , piece_id=E43SWVMVKWDULRK4YV6SDVJBCHWSAHSFRNACXSZZZQM2IU5JJ4KA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322937532 (743h39m9.919708565s) [noise=true , address=193.149.164.70:4105 , piece_id=7CKN7FHVF6OXY6ZPFZPM4DYNDOJNHGGF3PF3VFM7V2FSTAURFGWA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.323051576 (743h39m9.919822591s) [noise=true , address=77.90.31.17:4775 , piece_id=CFR7LFL2JZCVX4N7X2QQPJVJLZDSD6NZCNLNLSUJKU4DIZ7BBL3Q, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.323474343 (743h39m9.920245373s) [noise=true , address=167.17.72.68:4665 , piece_id=H7Y37VGEKXJGRPEZ2G4NG6WXTNNJ6YQMQI4NRUZYJTWMWIVF246Q, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.386273803 (743h39m9.983044815s) [upload=832686]: Next returned nil segment Apr 6 10:39:12.386288014 (743h39m9.983059037s) [upload=832686]: Waiting for error group managing segments... Apr 6 10:39:12.392899406 (743h39m9.989670418s) [noise=true , address=77.90.31.17:4775 , piece_id=CFR7LFL2JZCVX4N7X2QQPJVJLZDSD6NZCNLNLSUJKU4DIZ7BBL3Q, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.393163291 (743h39m9.989934315s) [noise=true , address=145.79.227.41:4116 , piece_id=E43SWVMVKWDULRK4YV6SDVJBCHWSAHSFRNACXSZZZQM2IU5JJ4KA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.667307989 (743h39m10.264079022s) [noise=true , address=167.17.72.68:4665 , piece_id=H7Y37VGEKXJGRPEZ2G4NG6WXTNNJ6YQMQI4NRUZYJTWMWIVF246Q, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.714497903 (743h39m10.311268937s) [noise=true , address=167.17.72.69:4734 , piece_id=I7HV3YRWA4IH7VE2I7SQFA42VHSYJVZCKA2KP4XQ4KSRJ6S7KPLA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.734199134 (743h39m10.330970147s) [noise=true , address=167.17.72.70:4761 , piece_id=WFSQTX3PFW77TZQCSIUSHSD3LH74XTFKLYUMDNYG7MFERLUCSZHA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.896708412 (743h39m10.493479439s) [noise=true , address=193.149.164.69:4104 , piece_id=Q3VXWW73OASMNUYH77EZBCQU5GAGPG3X3TKRLED43F3R6MZYZ2DQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.899988929 (743h39m10.496759964s) [noise=true , address=193.149.164.67:4008 , piece_id=SJM4PFZHQJW7I5C3XWXIIAJF7DBXPQQYGHTB22AB7JKMG6VN7QZA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.908433760 (743h39m10.505204783s) [noise=true , address=193.149.164.70:4105 , piece_id=7CKN7FHVF6OXY6ZPFZPM4DYNDOJNHGGF3PF3VFM7V2FSTAURFGWA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.916130090 (743h39m10.512901122s) [noise=true , address=193.149.164.68:4055 , piece_id=ARLLKHKO3AQJG52QH6EWHE55IKPQIEYCIIOR4RMIMU55TGYXYGKQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.956247905 (743h39m10.553018928s) [noise=true , address=193.149.164.66:4985 , piece_id=A6YQDCH7PTS6CP3IFWYMQ3HJGKFX6OSYQBATKUAIOXELHAEE5WBA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:13.000961275 (743h39m10.597732326s) [noise=true , address=193.149.164.143:4694 , piece_id=GG3FQN4YC5QQVGFCFYWTZWOZPFQO5NL2XE4FJCSUKUCTY4L5EWMA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:13.304880608 (743h39m10.90165162s) [noise=true , address=145.79.227.33:4878 , piece_id=YAEEULYYAHZP3RM6YAE55LC7S7DQLVV6JWIGYCY64GF6ITHJRQSQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:13.307202541 (743h39m10.903973568s) [noise=true , address=145.79.227.37:4582 , piece_id=QFSMYYEUDOO5JURWQHBLBRLCOXYSP4M2OR4OBWKIZ5LCCXPXEJCQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:40:12.257884464 (743h40m9.854655494s) [noise=true , address=45.91.116.12:4257 , piece_id=GW7MFYT77HZYDY263WMUIABDAELV37MBKMBS57FFSFXCIUJ5JN7A, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:12ZQ5a1TwxdeuJ6UwsVm9VsWnpZEBKC6N5yYUjyNTBkriSjvYvE, address:45.91.116.12:4257): protocol: rpc: dial tcp 45.91.116.12:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258122811 (743h40m9.854893836s) [noise=true , address=45.91.116.9:4270 , piece_id=UZYXLU235ANTBCMIIQDUHM6LZUKAS64FUBJIY2NXGYTIAO776KFQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:1Vr5WmL6Gem29LQTsCwQNFuqw6j4b1nj35AgbUWDA2Lyxf43YL, address:45.91.116.9:4270): protocol: read tcp 159.223.143.39:53356->45.91.116.9:4270: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258358693 (743h40m9.855129725s) [noise=true , address=45.91.116.6:4987 , piece_id=N2OUZHWZMNVIMAPTOHLIIT6NK5N3PVSHS6OX4ZGYSRUJIS5X3CDQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:12vPKcqLhopcpFxZ1LXwD8JXPLJB8a6HKfGnoyXsvKaL2oWUNao, address:45.91.116.6:4987): protocol: context deadline exceeded; context deadline exceeded Apr 6 10:40:12.258544102 (743h40m9.855315131s) [noise=true , address=45.91.116.11:4142 , piece_id=YAVLJVKDZWHWPN572Y5TX55ONWE2IDXVA6ABUF6KD7YRWCWR3YSQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:135mYyE4Gy1gcCXSocnb41e9u4z4MpqMjodmaRnrNs9AGS1YcX, address:45.91.116.11:4142): protocol: write tcp 159.223.143.39:34030->45.91.116.11:4142: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258881055 (743h40m9.855652086s) [noise=true , address=45.91.116.13:4166 , piece_id=CNVYP6YUT7E3VUCCSUR2ZWCDYU6QMOZL5GPZAMPQ5P2BVSD6HUDQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:12ajyozgSuf68kn8sUuEmuSfYCcxzU8Cwmg1EXVt9WV6dKSopy7, address:45.91.116.13:4166): protocol: write tcp 159.223.143.39:45960->45.91.116.13:4166: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258934014 (743h40m9.855705039s) [seg_pos={0 0}, upload=832686]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 10:40:12.278983295 (743h40m9.875754318s) [upload=832686]: Done uploading segments... Apr 6 10:40:12.279076909 (743h40m9.875847939s) [upload=832686]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.625920101s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 568.171098ms trace: https://jaeger.dev.storj.io/trace/7756830e434d8498 FAIL (2026-04-06 11:22:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 568.270039ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 1.297887743s trace: https://jaeger.dev.storj.io/trace/7e44fa01de290116 FAIL (2026-04-06 11:22:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== loading segments... Apr 6 10:39:12.293175658 (743h39m9.88994668s) [upload=832686]: Got next segment. Inline: false Apr 6 10:39:12.293210466 (743h39m9.889981489s) [upload=832686]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 10:39:12.293215002 (743h39m9.889986025s) [upload=832686]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 10:39:12.319174409 (743h39m9.915945433s) [seg_pos={0 0}, upload=832686]: Begin upload segment... Apr 6 10:39:12.319264769 (743h39m9.916035785s) [seg_pos={0 0}, upload=832686]: Done begin upload segment. Apr 6 10:39:12.319310748 (743h39m9.91608178s) [noise=true , address=45.91.116.13:4166 , piece_id=CNVYP6YUT7E3VUCCSUR2ZWCDYU6QMOZL5GPZAMPQ5P2BVSD6HUDQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.319344730 (743h39m9.916115762s) [noise=true , address=45.91.116.6:4987 , piece_id=N2OUZHWZMNVIMAPTOHLIIT6NK5N3PVSHS6OX4ZGYSRUJIS5X3CDQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.319839277 (743h39m9.916610309s) [noise=true , address=193.149.164.67:4008 , piece_id=SJM4PFZHQJW7I5C3XWXIIAJF7DBXPQQYGHTB22AB7JKMG6VN7QZA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320290481 (743h39m9.917061503s) [noise=true , address=167.17.72.69:4734 , piece_id=I7HV3YRWA4IH7VE2I7SQFA42VHSYJVZCKA2KP4XQ4KSRJ6S7KPLA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320450536 (743h39m9.917221562s) [noise=true , address=45.91.116.9:4270 , piece_id=UZYXLU235ANTBCMIIQDUHM6LZUKAS64FUBJIY2NXGYTIAO776KFQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320733701 (743h39m9.917504727s) [noise=true , address=193.149.164.143:4694 , piece_id=GG3FQN4YC5QQVGFCFYWTZWOZPFQO5NL2XE4FJCSUKUCTY4L5EWMA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320988084 (743h39m9.917759097s) [noise=true , address=45.91.116.12:4257 , piece_id=GW7MFYT77HZYDY263WMUIABDAELV37MBKMBS57FFSFXCIUJ5JN7A, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321148099 (743h39m9.917919124s) [noise=true , address=145.79.227.37:4582 , piece_id=QFSMYYEUDOO5JURWQHBLBRLCOXYSP4M2OR4OBWKIZ5LCCXPXEJCQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321494813 (743h39m9.918265836s) [noise=true , address=45.91.116.11:4142 , piece_id=YAVLJVKDZWHWPN572Y5TX55ONWE2IDXVA6ABUF6KD7YRWCWR3YSQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321531682 (743h39m9.918302714s) [noise=true , address=193.149.164.68:4055 , piece_id=ARLLKHKO3AQJG52QH6EWHE55IKPQIEYCIIOR4RMIMU55TGYXYGKQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321879088 (743h39m9.918650182s) [noise=true , address=193.149.164.69:4104 , piece_id=Q3VXWW73OASMNUYH77EZBCQU5GAGPG3X3TKRLED43F3R6MZYZ2DQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322027388 (743h39m9.918798417s) [noise=true , address=193.149.164.66:4985 , piece_id=A6YQDCH7PTS6CP3IFWYMQ3HJGKFX6OSYQBATKUAIOXELHAEE5WBA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322241446 (743h39m9.919012475s) [noise=true , address=145.79.227.33:4878 , piece_id=YAEEULYYAHZP3RM6YAE55LC7S7DQLVV6JWIGYCY64GF6ITHJRQSQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322546472 (743h39m9.919317502s) [noise=true , address=167.17.72.70:4761 , piece_id=WFSQTX3PFW77TZQCSIUSHSD3LH74XTFKLYUMDNYG7MFERLUCSZHA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322583845 (743h39m9.919354868s) [noise=true , address=145.79.227.41:4116 , piece_id=E43SWVMVKWDULRK4YV6SDVJBCHWSAHSFRNACXSZZZQM2IU5JJ4KA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322937532 (743h39m9.919708565s) [noise=true , address=193.149.164.70:4105 , piece_id=7CKN7FHVF6OXY6ZPFZPM4DYNDOJNHGGF3PF3VFM7V2FSTAURFGWA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.323051576 (743h39m9.919822591s) [noise=true , address=77.90.31.17:4775 , piece_id=CFR7LFL2JZCVX4N7X2QQPJVJLZDSD6NZCNLNLSUJKU4DIZ7BBL3Q, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.323474343 (743h39m9.920245373s) [noise=true , address=167.17.72.68:4665 , piece_id=H7Y37VGEKXJGRPEZ2G4NG6WXTNNJ6YQMQI4NRUZYJTWMWIVF246Q, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.386273803 (743h39m9.983044815s) [upload=832686]: Next returned nil segment Apr 6 10:39:12.386288014 (743h39m9.983059037s) [upload=832686]: Waiting for error group managing segments... Apr 6 10:39:12.392899406 (743h39m9.989670418s) [noise=true , address=77.90.31.17:4775 , piece_id=CFR7LFL2JZCVX4N7X2QQPJVJLZDSD6NZCNLNLSUJKU4DIZ7BBL3Q, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.393163291 (743h39m9.989934315s) [noise=true , address=145.79.227.41:4116 , piece_id=E43SWVMVKWDULRK4YV6SDVJBCHWSAHSFRNACXSZZZQM2IU5JJ4KA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.667307989 (743h39m10.264079022s) [noise=true , address=167.17.72.68:4665 , piece_id=H7Y37VGEKXJGRPEZ2G4NG6WXTNNJ6YQMQI4NRUZYJTWMWIVF246Q, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.714497903 (743h39m10.311268937s) [noise=true , address=167.17.72.69:4734 , piece_id=I7HV3YRWA4IH7VE2I7SQFA42VHSYJVZCKA2KP4XQ4KSRJ6S7KPLA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.734199134 (743h39m10.330970147s) [noise=true , address=167.17.72.70:4761 , piece_id=WFSQTX3PFW77TZQCSIUSHSD3LH74XTFKLYUMDNYG7MFERLUCSZHA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.896708412 (743h39m10.493479439s) [noise=true , address=193.149.164.69:4104 , piece_id=Q3VXWW73OASMNUYH77EZBCQU5GAGPG3X3TKRLED43F3R6MZYZ2DQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.899988929 (743h39m10.496759964s) [noise=true , address=193.149.164.67:4008 , piece_id=SJM4PFZHQJW7I5C3XWXIIAJF7DBXPQQYGHTB22AB7JKMG6VN7QZA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.908433760 (743h39m10.505204783s) [noise=true , address=193.149.164.70:4105 , piece_id=7CKN7FHVF6OXY6ZPFZPM4DYNDOJNHGGF3PF3VFM7V2FSTAURFGWA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.916130090 (743h39m10.512901122s) [noise=true , address=193.149.164.68:4055 , piece_id=ARLLKHKO3AQJG52QH6EWHE55IKPQIEYCIIOR4RMIMU55TGYXYGKQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.956247905 (743h39m10.553018928s) [noise=true , address=193.149.164.66:4985 , piece_id=A6YQDCH7PTS6CP3IFWYMQ3HJGKFX6OSYQBATKUAIOXELHAEE5WBA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:13.000961275 (743h39m10.597732326s) [noise=true , address=193.149.164.143:4694 , piece_id=GG3FQN4YC5QQVGFCFYWTZWOZPFQO5NL2XE4FJCSUKUCTY4L5EWMA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:13.304880608 (743h39m10.90165162s) [noise=true , address=145.79.227.33:4878 , piece_id=YAEEULYYAHZP3RM6YAE55LC7S7DQLVV6JWIGYCY64GF6ITHJRQSQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:13.307202541 (743h39m10.903973568s) [noise=true , address=145.79.227.37:4582 , piece_id=QFSMYYEUDOO5JURWQHBLBRLCOXYSP4M2OR4OBWKIZ5LCCXPXEJCQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:40:12.257884464 (743h40m9.854655494s) [noise=true , address=45.91.116.12:4257 , piece_id=GW7MFYT77HZYDY263WMUIABDAELV37MBKMBS57FFSFXCIUJ5JN7A, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:12ZQ5a1TwxdeuJ6UwsVm9VsWnpZEBKC6N5yYUjyNTBkriSjvYvE, address:45.91.116.12:4257): protocol: rpc: dial tcp 45.91.116.12:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258122811 (743h40m9.854893836s) [noise=true , address=45.91.116.9:4270 , piece_id=UZYXLU235ANTBCMIIQDUHM6LZUKAS64FUBJIY2NXGYTIAO776KFQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:1Vr5WmL6Gem29LQTsCwQNFuqw6j4b1nj35AgbUWDA2Lyxf43YL, address:45.91.116.9:4270): protocol: read tcp 159.223.143.39:53356->45.91.116.9:4270: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258358693 (743h40m9.855129725s) [noise=true , address=45.91.116.6:4987 , piece_id=N2OUZHWZMNVIMAPTOHLIIT6NK5N3PVSHS6OX4ZGYSRUJIS5X3CDQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:12vPKcqLhopcpFxZ1LXwD8JXPLJB8a6HKfGnoyXsvKaL2oWUNao, address:45.91.116.6:4987): protocol: context deadline exceeded; context deadline exceeded Apr 6 10:40:12.258544102 (743h40m9.855315131s) [noise=true , address=45.91.116.11:4142 , piece_id=YAVLJVKDZWHWPN572Y5TX55ONWE2IDXVA6ABUF6KD7YRWCWR3YSQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:135mYyE4Gy1gcCXSocnb41e9u4z4MpqMjodmaRnrNs9AGS1YcX, address:45.91.116.11:4142): protocol: write tcp 159.223.143.39:34030->45.91.116.11:4142: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258881055 (743h40m9.855652086s) [noise=true , address=45.91.116.13:4166 , piece_id=CNVYP6YUT7E3VUCCSUR2ZWCDYU6QMOZL5GPZAMPQ5P2BVSD6HUDQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:12ajyozgSuf68kn8sUuEmuSfYCcxzU8Cwmg1EXVt9WV6dKSopy7, address:45.91.116.13:4166): protocol: write tcp 159.223.143.39:45960->45.91.116.13:4166: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258934014 (743h40m9.855705039s) [seg_pos={0 0}, upload=832686]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 10:40:12.278983295 (743h40m9.875754318s) [upload=832686]: Done uploading segments... Apr 6 10:40:12.279076909 (743h40m9.875847939s) [upload=832686]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.297940129s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [..XXX.......................................................] duration: 513.736537ms trace: https://jaeger.dev.storj.io/trace/4736d76b51c361a3 FAIL (2026-04-06 11:22:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 513.773849ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 728.277539ms trace: https://jaeger.dev.storj.io/trace/7756830e434d8498 FAIL (2026-04-06 11:21:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 728.327278ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 1.356464502s trace: https://jaeger.dev.storj.io/trace/7e44fa01de290116 FAIL (2026-04-06 11:21:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== loading segments... Apr 6 10:39:12.293175658 (743h39m9.88994668s) [upload=832686]: Got next segment. Inline: false Apr 6 10:39:12.293210466 (743h39m9.889981489s) [upload=832686]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 10:39:12.293215002 (743h39m9.889986025s) [upload=832686]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 10:39:12.319174409 (743h39m9.915945433s) [seg_pos={0 0}, upload=832686]: Begin upload segment... Apr 6 10:39:12.319264769 (743h39m9.916035785s) [seg_pos={0 0}, upload=832686]: Done begin upload segment. Apr 6 10:39:12.319310748 (743h39m9.91608178s) [noise=true , address=45.91.116.13:4166 , piece_id=CNVYP6YUT7E3VUCCSUR2ZWCDYU6QMOZL5GPZAMPQ5P2BVSD6HUDQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.319344730 (743h39m9.916115762s) [noise=true , address=45.91.116.6:4987 , piece_id=N2OUZHWZMNVIMAPTOHLIIT6NK5N3PVSHS6OX4ZGYSRUJIS5X3CDQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.319839277 (743h39m9.916610309s) [noise=true , address=193.149.164.67:4008 , piece_id=SJM4PFZHQJW7I5C3XWXIIAJF7DBXPQQYGHTB22AB7JKMG6VN7QZA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320290481 (743h39m9.917061503s) [noise=true , address=167.17.72.69:4734 , piece_id=I7HV3YRWA4IH7VE2I7SQFA42VHSYJVZCKA2KP4XQ4KSRJ6S7KPLA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320450536 (743h39m9.917221562s) [noise=true , address=45.91.116.9:4270 , piece_id=UZYXLU235ANTBCMIIQDUHM6LZUKAS64FUBJIY2NXGYTIAO776KFQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320733701 (743h39m9.917504727s) [noise=true , address=193.149.164.143:4694 , piece_id=GG3FQN4YC5QQVGFCFYWTZWOZPFQO5NL2XE4FJCSUKUCTY4L5EWMA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320988084 (743h39m9.917759097s) [noise=true , address=45.91.116.12:4257 , piece_id=GW7MFYT77HZYDY263WMUIABDAELV37MBKMBS57FFSFXCIUJ5JN7A, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321148099 (743h39m9.917919124s) [noise=true , address=145.79.227.37:4582 , piece_id=QFSMYYEUDOO5JURWQHBLBRLCOXYSP4M2OR4OBWKIZ5LCCXPXEJCQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321494813 (743h39m9.918265836s) [noise=true , address=45.91.116.11:4142 , piece_id=YAVLJVKDZWHWPN572Y5TX55ONWE2IDXVA6ABUF6KD7YRWCWR3YSQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321531682 (743h39m9.918302714s) [noise=true , address=193.149.164.68:4055 , piece_id=ARLLKHKO3AQJG52QH6EWHE55IKPQIEYCIIOR4RMIMU55TGYXYGKQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321879088 (743h39m9.918650182s) [noise=true , address=193.149.164.69:4104 , piece_id=Q3VXWW73OASMNUYH77EZBCQU5GAGPG3X3TKRLED43F3R6MZYZ2DQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322027388 (743h39m9.918798417s) [noise=true , address=193.149.164.66:4985 , piece_id=A6YQDCH7PTS6CP3IFWYMQ3HJGKFX6OSYQBATKUAIOXELHAEE5WBA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322241446 (743h39m9.919012475s) [noise=true , address=145.79.227.33:4878 , piece_id=YAEEULYYAHZP3RM6YAE55LC7S7DQLVV6JWIGYCY64GF6ITHJRQSQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322546472 (743h39m9.919317502s) [noise=true , address=167.17.72.70:4761 , piece_id=WFSQTX3PFW77TZQCSIUSHSD3LH74XTFKLYUMDNYG7MFERLUCSZHA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322583845 (743h39m9.919354868s) [noise=true , address=145.79.227.41:4116 , piece_id=E43SWVMVKWDULRK4YV6SDVJBCHWSAHSFRNACXSZZZQM2IU5JJ4KA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322937532 (743h39m9.919708565s) [noise=true , address=193.149.164.70:4105 , piece_id=7CKN7FHVF6OXY6ZPFZPM4DYNDOJNHGGF3PF3VFM7V2FSTAURFGWA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.323051576 (743h39m9.919822591s) [noise=true , address=77.90.31.17:4775 , piece_id=CFR7LFL2JZCVX4N7X2QQPJVJLZDSD6NZCNLNLSUJKU4DIZ7BBL3Q, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.323474343 (743h39m9.920245373s) [noise=true , address=167.17.72.68:4665 , piece_id=H7Y37VGEKXJGRPEZ2G4NG6WXTNNJ6YQMQI4NRUZYJTWMWIVF246Q, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.386273803 (743h39m9.983044815s) [upload=832686]: Next returned nil segment Apr 6 10:39:12.386288014 (743h39m9.983059037s) [upload=832686]: Waiting for error group managing segments... Apr 6 10:39:12.392899406 (743h39m9.989670418s) [noise=true , address=77.90.31.17:4775 , piece_id=CFR7LFL2JZCVX4N7X2QQPJVJLZDSD6NZCNLNLSUJKU4DIZ7BBL3Q, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.393163291 (743h39m9.989934315s) [noise=true , address=145.79.227.41:4116 , piece_id=E43SWVMVKWDULRK4YV6SDVJBCHWSAHSFRNACXSZZZQM2IU5JJ4KA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.667307989 (743h39m10.264079022s) [noise=true , address=167.17.72.68:4665 , piece_id=H7Y37VGEKXJGRPEZ2G4NG6WXTNNJ6YQMQI4NRUZYJTWMWIVF246Q, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.714497903 (743h39m10.311268937s) [noise=true , address=167.17.72.69:4734 , piece_id=I7HV3YRWA4IH7VE2I7SQFA42VHSYJVZCKA2KP4XQ4KSRJ6S7KPLA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.734199134 (743h39m10.330970147s) [noise=true , address=167.17.72.70:4761 , piece_id=WFSQTX3PFW77TZQCSIUSHSD3LH74XTFKLYUMDNYG7MFERLUCSZHA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.896708412 (743h39m10.493479439s) [noise=true , address=193.149.164.69:4104 , piece_id=Q3VXWW73OASMNUYH77EZBCQU5GAGPG3X3TKRLED43F3R6MZYZ2DQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.899988929 (743h39m10.496759964s) [noise=true , address=193.149.164.67:4008 , piece_id=SJM4PFZHQJW7I5C3XWXIIAJF7DBXPQQYGHTB22AB7JKMG6VN7QZA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.908433760 (743h39m10.505204783s) [noise=true , address=193.149.164.70:4105 , piece_id=7CKN7FHVF6OXY6ZPFZPM4DYNDOJNHGGF3PF3VFM7V2FSTAURFGWA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.916130090 (743h39m10.512901122s) [noise=true , address=193.149.164.68:4055 , piece_id=ARLLKHKO3AQJG52QH6EWHE55IKPQIEYCIIOR4RMIMU55TGYXYGKQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.956247905 (743h39m10.553018928s) [noise=true , address=193.149.164.66:4985 , piece_id=A6YQDCH7PTS6CP3IFWYMQ3HJGKFX6OSYQBATKUAIOXELHAEE5WBA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:13.000961275 (743h39m10.597732326s) [noise=true , address=193.149.164.143:4694 , piece_id=GG3FQN4YC5QQVGFCFYWTZWOZPFQO5NL2XE4FJCSUKUCTY4L5EWMA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:13.304880608 (743h39m10.90165162s) [noise=true , address=145.79.227.33:4878 , piece_id=YAEEULYYAHZP3RM6YAE55LC7S7DQLVV6JWIGYCY64GF6ITHJRQSQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:13.307202541 (743h39m10.903973568s) [noise=true , address=145.79.227.37:4582 , piece_id=QFSMYYEUDOO5JURWQHBLBRLCOXYSP4M2OR4OBWKIZ5LCCXPXEJCQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:40:12.257884464 (743h40m9.854655494s) [noise=true , address=45.91.116.12:4257 , piece_id=GW7MFYT77HZYDY263WMUIABDAELV37MBKMBS57FFSFXCIUJ5JN7A, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:12ZQ5a1TwxdeuJ6UwsVm9VsWnpZEBKC6N5yYUjyNTBkriSjvYvE, address:45.91.116.12:4257): protocol: rpc: dial tcp 45.91.116.12:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258122811 (743h40m9.854893836s) [noise=true , address=45.91.116.9:4270 , piece_id=UZYXLU235ANTBCMIIQDUHM6LZUKAS64FUBJIY2NXGYTIAO776KFQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:1Vr5WmL6Gem29LQTsCwQNFuqw6j4b1nj35AgbUWDA2Lyxf43YL, address:45.91.116.9:4270): protocol: read tcp 159.223.143.39:53356->45.91.116.9:4270: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258358693 (743h40m9.855129725s) [noise=true , address=45.91.116.6:4987 , piece_id=N2OUZHWZMNVIMAPTOHLIIT6NK5N3PVSHS6OX4ZGYSRUJIS5X3CDQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:12vPKcqLhopcpFxZ1LXwD8JXPLJB8a6HKfGnoyXsvKaL2oWUNao, address:45.91.116.6:4987): protocol: context deadline exceeded; context deadline exceeded Apr 6 10:40:12.258544102 (743h40m9.855315131s) [noise=true , address=45.91.116.11:4142 , piece_id=YAVLJVKDZWHWPN572Y5TX55ONWE2IDXVA6ABUF6KD7YRWCWR3YSQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:135mYyE4Gy1gcCXSocnb41e9u4z4MpqMjodmaRnrNs9AGS1YcX, address:45.91.116.11:4142): protocol: write tcp 159.223.143.39:34030->45.91.116.11:4142: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258881055 (743h40m9.855652086s) [noise=true , address=45.91.116.13:4166 , piece_id=CNVYP6YUT7E3VUCCSUR2ZWCDYU6QMOZL5GPZAMPQ5P2BVSD6HUDQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:12ajyozgSuf68kn8sUuEmuSfYCcxzU8Cwmg1EXVt9WV6dKSopy7, address:45.91.116.13:4166): protocol: write tcp 159.223.143.39:45960->45.91.116.13:4166: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258934014 (743h40m9.855705039s) [seg_pos={0 0}, upload=832686]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 10:40:12.278983295 (743h40m9.875754318s) [upload=832686]: Done uploading segments... Apr 6 10:40:12.279076909 (743h40m9.875847939s) [upload=832686]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.356501838s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 595.5543ms trace: https://jaeger.dev.storj.io/trace/7756830e434d8498 FAIL (2026-04-06 11:21:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 595.652826ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [...XXX......................................................] duration: 457.607098ms trace: https://jaeger.dev.storj.io/trace/4736d76b51c361a3 FAIL (2026-04-06 11:21:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 457.654073ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [....XXX.....................................................] duration: 476.133064ms trace: https://jaeger.dev.storj.io/trace/4736d76b51c361a3 FAIL (2026-04-06 11:20:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 476.167303ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 660.164508ms trace: https://jaeger.dev.storj.io/trace/7756830e434d8498 FAIL (2026-04-06 11:20:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 660.192878ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 1.358584152s trace: https://jaeger.dev.storj.io/trace/7e44fa01de290116 FAIL (2026-04-06 11:20:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== loading segments... Apr 6 10:39:12.293175658 (743h39m9.88994668s) [upload=832686]: Got next segment. Inline: false Apr 6 10:39:12.293210466 (743h39m9.889981489s) [upload=832686]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 10:39:12.293215002 (743h39m9.889986025s) [upload=832686]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 10:39:12.319174409 (743h39m9.915945433s) [seg_pos={0 0}, upload=832686]: Begin upload segment... Apr 6 10:39:12.319264769 (743h39m9.916035785s) [seg_pos={0 0}, upload=832686]: Done begin upload segment. Apr 6 10:39:12.319310748 (743h39m9.91608178s) [noise=true , address=45.91.116.13:4166 , piece_id=CNVYP6YUT7E3VUCCSUR2ZWCDYU6QMOZL5GPZAMPQ5P2BVSD6HUDQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.319344730 (743h39m9.916115762s) [noise=true , address=45.91.116.6:4987 , piece_id=N2OUZHWZMNVIMAPTOHLIIT6NK5N3PVSHS6OX4ZGYSRUJIS5X3CDQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.319839277 (743h39m9.916610309s) [noise=true , address=193.149.164.67:4008 , piece_id=SJM4PFZHQJW7I5C3XWXIIAJF7DBXPQQYGHTB22AB7JKMG6VN7QZA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320290481 (743h39m9.917061503s) [noise=true , address=167.17.72.69:4734 , piece_id=I7HV3YRWA4IH7VE2I7SQFA42VHSYJVZCKA2KP4XQ4KSRJ6S7KPLA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320450536 (743h39m9.917221562s) [noise=true , address=45.91.116.9:4270 , piece_id=UZYXLU235ANTBCMIIQDUHM6LZUKAS64FUBJIY2NXGYTIAO776KFQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320733701 (743h39m9.917504727s) [noise=true , address=193.149.164.143:4694 , piece_id=GG3FQN4YC5QQVGFCFYWTZWOZPFQO5NL2XE4FJCSUKUCTY4L5EWMA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320988084 (743h39m9.917759097s) [noise=true , address=45.91.116.12:4257 , piece_id=GW7MFYT77HZYDY263WMUIABDAELV37MBKMBS57FFSFXCIUJ5JN7A, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321148099 (743h39m9.917919124s) [noise=true , address=145.79.227.37:4582 , piece_id=QFSMYYEUDOO5JURWQHBLBRLCOXYSP4M2OR4OBWKIZ5LCCXPXEJCQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321494813 (743h39m9.918265836s) [noise=true , address=45.91.116.11:4142 , piece_id=YAVLJVKDZWHWPN572Y5TX55ONWE2IDXVA6ABUF6KD7YRWCWR3YSQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321531682 (743h39m9.918302714s) [noise=true , address=193.149.164.68:4055 , piece_id=ARLLKHKO3AQJG52QH6EWHE55IKPQIEYCIIOR4RMIMU55TGYXYGKQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321879088 (743h39m9.918650182s) [noise=true , address=193.149.164.69:4104 , piece_id=Q3VXWW73OASMNUYH77EZBCQU5GAGPG3X3TKRLED43F3R6MZYZ2DQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322027388 (743h39m9.918798417s) [noise=true , address=193.149.164.66:4985 , piece_id=A6YQDCH7PTS6CP3IFWYMQ3HJGKFX6OSYQBATKUAIOXELHAEE5WBA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322241446 (743h39m9.919012475s) [noise=true , address=145.79.227.33:4878 , piece_id=YAEEULYYAHZP3RM6YAE55LC7S7DQLVV6JWIGYCY64GF6ITHJRQSQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322546472 (743h39m9.919317502s) [noise=true , address=167.17.72.70:4761 , piece_id=WFSQTX3PFW77TZQCSIUSHSD3LH74XTFKLYUMDNYG7MFERLUCSZHA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322583845 (743h39m9.919354868s) [noise=true , address=145.79.227.41:4116 , piece_id=E43SWVMVKWDULRK4YV6SDVJBCHWSAHSFRNACXSZZZQM2IU5JJ4KA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322937532 (743h39m9.919708565s) [noise=true , address=193.149.164.70:4105 , piece_id=7CKN7FHVF6OXY6ZPFZPM4DYNDOJNHGGF3PF3VFM7V2FSTAURFGWA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.323051576 (743h39m9.919822591s) [noise=true , address=77.90.31.17:4775 , piece_id=CFR7LFL2JZCVX4N7X2QQPJVJLZDSD6NZCNLNLSUJKU4DIZ7BBL3Q, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.323474343 (743h39m9.920245373s) [noise=true , address=167.17.72.68:4665 , piece_id=H7Y37VGEKXJGRPEZ2G4NG6WXTNNJ6YQMQI4NRUZYJTWMWIVF246Q, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.386273803 (743h39m9.983044815s) [upload=832686]: Next returned nil segment Apr 6 10:39:12.386288014 (743h39m9.983059037s) [upload=832686]: Waiting for error group managing segments... Apr 6 10:39:12.392899406 (743h39m9.989670418s) [noise=true , address=77.90.31.17:4775 , piece_id=CFR7LFL2JZCVX4N7X2QQPJVJLZDSD6NZCNLNLSUJKU4DIZ7BBL3Q, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.393163291 (743h39m9.989934315s) [noise=true , address=145.79.227.41:4116 , piece_id=E43SWVMVKWDULRK4YV6SDVJBCHWSAHSFRNACXSZZZQM2IU5JJ4KA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.667307989 (743h39m10.264079022s) [noise=true , address=167.17.72.68:4665 , piece_id=H7Y37VGEKXJGRPEZ2G4NG6WXTNNJ6YQMQI4NRUZYJTWMWIVF246Q, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.714497903 (743h39m10.311268937s) [noise=true , address=167.17.72.69:4734 , piece_id=I7HV3YRWA4IH7VE2I7SQFA42VHSYJVZCKA2KP4XQ4KSRJ6S7KPLA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.734199134 (743h39m10.330970147s) [noise=true , address=167.17.72.70:4761 , piece_id=WFSQTX3PFW77TZQCSIUSHSD3LH74XTFKLYUMDNYG7MFERLUCSZHA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.896708412 (743h39m10.493479439s) [noise=true , address=193.149.164.69:4104 , piece_id=Q3VXWW73OASMNUYH77EZBCQU5GAGPG3X3TKRLED43F3R6MZYZ2DQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.899988929 (743h39m10.496759964s) [noise=true , address=193.149.164.67:4008 , piece_id=SJM4PFZHQJW7I5C3XWXIIAJF7DBXPQQYGHTB22AB7JKMG6VN7QZA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.908433760 (743h39m10.505204783s) [noise=true , address=193.149.164.70:4105 , piece_id=7CKN7FHVF6OXY6ZPFZPM4DYNDOJNHGGF3PF3VFM7V2FSTAURFGWA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.916130090 (743h39m10.512901122s) [noise=true , address=193.149.164.68:4055 , piece_id=ARLLKHKO3AQJG52QH6EWHE55IKPQIEYCIIOR4RMIMU55TGYXYGKQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.956247905 (743h39m10.553018928s) [noise=true , address=193.149.164.66:4985 , piece_id=A6YQDCH7PTS6CP3IFWYMQ3HJGKFX6OSYQBATKUAIOXELHAEE5WBA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:13.000961275 (743h39m10.597732326s) [noise=true , address=193.149.164.143:4694 , piece_id=GG3FQN4YC5QQVGFCFYWTZWOZPFQO5NL2XE4FJCSUKUCTY4L5EWMA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:13.304880608 (743h39m10.90165162s) [noise=true , address=145.79.227.33:4878 , piece_id=YAEEULYYAHZP3RM6YAE55LC7S7DQLVV6JWIGYCY64GF6ITHJRQSQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:13.307202541 (743h39m10.903973568s) [noise=true , address=145.79.227.37:4582 , piece_id=QFSMYYEUDOO5JURWQHBLBRLCOXYSP4M2OR4OBWKIZ5LCCXPXEJCQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:40:12.257884464 (743h40m9.854655494s) [noise=true , address=45.91.116.12:4257 , piece_id=GW7MFYT77HZYDY263WMUIABDAELV37MBKMBS57FFSFXCIUJ5JN7A, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:12ZQ5a1TwxdeuJ6UwsVm9VsWnpZEBKC6N5yYUjyNTBkriSjvYvE, address:45.91.116.12:4257): protocol: rpc: dial tcp 45.91.116.12:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258122811 (743h40m9.854893836s) [noise=true , address=45.91.116.9:4270 , piece_id=UZYXLU235ANTBCMIIQDUHM6LZUKAS64FUBJIY2NXGYTIAO776KFQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:1Vr5WmL6Gem29LQTsCwQNFuqw6j4b1nj35AgbUWDA2Lyxf43YL, address:45.91.116.9:4270): protocol: read tcp 159.223.143.39:53356->45.91.116.9:4270: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258358693 (743h40m9.855129725s) [noise=true , address=45.91.116.6:4987 , piece_id=N2OUZHWZMNVIMAPTOHLIIT6NK5N3PVSHS6OX4ZGYSRUJIS5X3CDQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:12vPKcqLhopcpFxZ1LXwD8JXPLJB8a6HKfGnoyXsvKaL2oWUNao, address:45.91.116.6:4987): protocol: context deadline exceeded; context deadline exceeded Apr 6 10:40:12.258544102 (743h40m9.855315131s) [noise=true , address=45.91.116.11:4142 , piece_id=YAVLJVKDZWHWPN572Y5TX55ONWE2IDXVA6ABUF6KD7YRWCWR3YSQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:135mYyE4Gy1gcCXSocnb41e9u4z4MpqMjodmaRnrNs9AGS1YcX, address:45.91.116.11:4142): protocol: write tcp 159.223.143.39:34030->45.91.116.11:4142: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258881055 (743h40m9.855652086s) [noise=true , address=45.91.116.13:4166 , piece_id=CNVYP6YUT7E3VUCCSUR2ZWCDYU6QMOZL5GPZAMPQ5P2BVSD6HUDQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:12ajyozgSuf68kn8sUuEmuSfYCcxzU8Cwmg1EXVt9WV6dKSopy7, address:45.91.116.13:4166): protocol: write tcp 159.223.143.39:45960->45.91.116.13:4166: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258934014 (743h40m9.855705039s) [seg_pos={0 0}, upload=832686]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 10:40:12.278983295 (743h40m9.875754318s) [upload=832686]: Done uploading segments... Apr 6 10:40:12.279076909 (743h40m9.875847939s) [upload=832686]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.358616538s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.....XXX....................................................] duration: 515.123773ms trace: https://jaeger.dev.storj.io/trace/4736d76b51c361a3 FAIL (2026-04-06 11:20:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 515.177707ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 1.018791508s trace: https://jaeger.dev.storj.io/trace/7756830e434d8498 FAIL (2026-04-06 11:20:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.019250227s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 1.27355026s trace: https://jaeger.dev.storj.io/trace/7e44fa01de290116 FAIL (2026-04-06 11:19:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== loading segments... Apr 6 10:39:12.293175658 (743h39m9.88994668s) [upload=832686]: Got next segment. Inline: false Apr 6 10:39:12.293210466 (743h39m9.889981489s) [upload=832686]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 6 10:39:12.293215002 (743h39m9.889986025s) [upload=832686]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 6 10:39:12.319174409 (743h39m9.915945433s) [seg_pos={0 0}, upload=832686]: Begin upload segment... Apr 6 10:39:12.319264769 (743h39m9.916035785s) [seg_pos={0 0}, upload=832686]: Done begin upload segment. Apr 6 10:39:12.319310748 (743h39m9.91608178s) [noise=true , address=45.91.116.13:4166 , piece_id=CNVYP6YUT7E3VUCCSUR2ZWCDYU6QMOZL5GPZAMPQ5P2BVSD6HUDQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.319344730 (743h39m9.916115762s) [noise=true , address=45.91.116.6:4987 , piece_id=N2OUZHWZMNVIMAPTOHLIIT6NK5N3PVSHS6OX4ZGYSRUJIS5X3CDQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.319839277 (743h39m9.916610309s) [noise=true , address=193.149.164.67:4008 , piece_id=SJM4PFZHQJW7I5C3XWXIIAJF7DBXPQQYGHTB22AB7JKMG6VN7QZA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320290481 (743h39m9.917061503s) [noise=true , address=167.17.72.69:4734 , piece_id=I7HV3YRWA4IH7VE2I7SQFA42VHSYJVZCKA2KP4XQ4KSRJ6S7KPLA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320450536 (743h39m9.917221562s) [noise=true , address=45.91.116.9:4270 , piece_id=UZYXLU235ANTBCMIIQDUHM6LZUKAS64FUBJIY2NXGYTIAO776KFQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320733701 (743h39m9.917504727s) [noise=true , address=193.149.164.143:4694 , piece_id=GG3FQN4YC5QQVGFCFYWTZWOZPFQO5NL2XE4FJCSUKUCTY4L5EWMA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.320988084 (743h39m9.917759097s) [noise=true , address=45.91.116.12:4257 , piece_id=GW7MFYT77HZYDY263WMUIABDAELV37MBKMBS57FFSFXCIUJ5JN7A, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321148099 (743h39m9.917919124s) [noise=true , address=145.79.227.37:4582 , piece_id=QFSMYYEUDOO5JURWQHBLBRLCOXYSP4M2OR4OBWKIZ5LCCXPXEJCQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321494813 (743h39m9.918265836s) [noise=true , address=45.91.116.11:4142 , piece_id=YAVLJVKDZWHWPN572Y5TX55ONWE2IDXVA6ABUF6KD7YRWCWR3YSQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321531682 (743h39m9.918302714s) [noise=true , address=193.149.164.68:4055 , piece_id=ARLLKHKO3AQJG52QH6EWHE55IKPQIEYCIIOR4RMIMU55TGYXYGKQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.321879088 (743h39m9.918650182s) [noise=true , address=193.149.164.69:4104 , piece_id=Q3VXWW73OASMNUYH77EZBCQU5GAGPG3X3TKRLED43F3R6MZYZ2DQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322027388 (743h39m9.918798417s) [noise=true , address=193.149.164.66:4985 , piece_id=A6YQDCH7PTS6CP3IFWYMQ3HJGKFX6OSYQBATKUAIOXELHAEE5WBA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322241446 (743h39m9.919012475s) [noise=true , address=145.79.227.33:4878 , piece_id=YAEEULYYAHZP3RM6YAE55LC7S7DQLVV6JWIGYCY64GF6ITHJRQSQ, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322546472 (743h39m9.919317502s) [noise=true , address=167.17.72.70:4761 , piece_id=WFSQTX3PFW77TZQCSIUSHSD3LH74XTFKLYUMDNYG7MFERLUCSZHA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322583845 (743h39m9.919354868s) [noise=true , address=145.79.227.41:4116 , piece_id=E43SWVMVKWDULRK4YV6SDVJBCHWSAHSFRNACXSZZZQM2IU5JJ4KA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.322937532 (743h39m9.919708565s) [noise=true , address=193.149.164.70:4105 , piece_id=7CKN7FHVF6OXY6ZPFZPM4DYNDOJNHGGF3PF3VFM7V2FSTAURFGWA, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.323051576 (743h39m9.919822591s) [noise=true , address=77.90.31.17:4775 , piece_id=CFR7LFL2JZCVX4N7X2QQPJVJLZDSD6NZCNLNLSUJKU4DIZ7BBL3Q, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.323474343 (743h39m9.920245373s) [noise=true , address=167.17.72.68:4665 , piece_id=H7Y37VGEKXJGRPEZ2G4NG6WXTNNJ6YQMQI4NRUZYJTWMWIVF246Q, seg_pos={0 0}, upload=832686]: Uploading piece... Apr 6 10:39:12.386273803 (743h39m9.983044815s) [upload=832686]: Next returned nil segment Apr 6 10:39:12.386288014 (743h39m9.983059037s) [upload=832686]: Waiting for error group managing segments... Apr 6 10:39:12.392899406 (743h39m9.989670418s) [noise=true , address=77.90.31.17:4775 , piece_id=CFR7LFL2JZCVX4N7X2QQPJVJLZDSD6NZCNLNLSUJKU4DIZ7BBL3Q, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.393163291 (743h39m9.989934315s) [noise=true , address=145.79.227.41:4116 , piece_id=E43SWVMVKWDULRK4YV6SDVJBCHWSAHSFRNACXSZZZQM2IU5JJ4KA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.667307989 (743h39m10.264079022s) [noise=true , address=167.17.72.68:4665 , piece_id=H7Y37VGEKXJGRPEZ2G4NG6WXTNNJ6YQMQI4NRUZYJTWMWIVF246Q, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.714497903 (743h39m10.311268937s) [noise=true , address=167.17.72.69:4734 , piece_id=I7HV3YRWA4IH7VE2I7SQFA42VHSYJVZCKA2KP4XQ4KSRJ6S7KPLA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.734199134 (743h39m10.330970147s) [noise=true , address=167.17.72.70:4761 , piece_id=WFSQTX3PFW77TZQCSIUSHSD3LH74XTFKLYUMDNYG7MFERLUCSZHA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.896708412 (743h39m10.493479439s) [noise=true , address=193.149.164.69:4104 , piece_id=Q3VXWW73OASMNUYH77EZBCQU5GAGPG3X3TKRLED43F3R6MZYZ2DQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.899988929 (743h39m10.496759964s) [noise=true , address=193.149.164.67:4008 , piece_id=SJM4PFZHQJW7I5C3XWXIIAJF7DBXPQQYGHTB22AB7JKMG6VN7QZA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.908433760 (743h39m10.505204783s) [noise=true , address=193.149.164.70:4105 , piece_id=7CKN7FHVF6OXY6ZPFZPM4DYNDOJNHGGF3PF3VFM7V2FSTAURFGWA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.916130090 (743h39m10.512901122s) [noise=true , address=193.149.164.68:4055 , piece_id=ARLLKHKO3AQJG52QH6EWHE55IKPQIEYCIIOR4RMIMU55TGYXYGKQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:12.956247905 (743h39m10.553018928s) [noise=true , address=193.149.164.66:4985 , piece_id=A6YQDCH7PTS6CP3IFWYMQ3HJGKFX6OSYQBATKUAIOXELHAEE5WBA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:13.000961275 (743h39m10.597732326s) [noise=true , address=193.149.164.143:4694 , piece_id=GG3FQN4YC5QQVGFCFYWTZWOZPFQO5NL2XE4FJCSUKUCTY4L5EWMA, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:13.304880608 (743h39m10.90165162s) [noise=true , address=145.79.227.33:4878 , piece_id=YAEEULYYAHZP3RM6YAE55LC7S7DQLVV6JWIGYCY64GF6ITHJRQSQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:39:13.307202541 (743h39m10.903973568s) [noise=true , address=145.79.227.37:4582 , piece_id=QFSMYYEUDOO5JURWQHBLBRLCOXYSP4M2OR4OBWKIZ5LCCXPXEJCQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: <nil> Apr 6 10:40:12.257884464 (743h40m9.854655494s) [noise=true , address=45.91.116.12:4257 , piece_id=GW7MFYT77HZYDY263WMUIABDAELV37MBKMBS57FFSFXCIUJ5JN7A, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:12ZQ5a1TwxdeuJ6UwsVm9VsWnpZEBKC6N5yYUjyNTBkriSjvYvE, address:45.91.116.12:4257): protocol: rpc: dial tcp 45.91.116.12:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258122811 (743h40m9.854893836s) [noise=true , address=45.91.116.9:4270 , piece_id=UZYXLU235ANTBCMIIQDUHM6LZUKAS64FUBJIY2NXGYTIAO776KFQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:1Vr5WmL6Gem29LQTsCwQNFuqw6j4b1nj35AgbUWDA2Lyxf43YL, address:45.91.116.9:4270): protocol: read tcp 159.223.143.39:53356->45.91.116.9:4270: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258358693 (743h40m9.855129725s) [noise=true , address=45.91.116.6:4987 , piece_id=N2OUZHWZMNVIMAPTOHLIIT6NK5N3PVSHS6OX4ZGYSRUJIS5X3CDQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:12vPKcqLhopcpFxZ1LXwD8JXPLJB8a6HKfGnoyXsvKaL2oWUNao, address:45.91.116.6:4987): protocol: context deadline exceeded; context deadline exceeded Apr 6 10:40:12.258544102 (743h40m9.855315131s) [noise=true , address=45.91.116.11:4142 , piece_id=YAVLJVKDZWHWPN572Y5TX55ONWE2IDXVA6ABUF6KD7YRWCWR3YSQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:135mYyE4Gy1gcCXSocnb41e9u4z4MpqMjodmaRnrNs9AGS1YcX, address:45.91.116.11:4142): protocol: write tcp 159.223.143.39:34030->45.91.116.11:4142: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258881055 (743h40m9.855652086s) [noise=true , address=45.91.116.13:4166 , piece_id=CNVYP6YUT7E3VUCCSUR2ZWCDYU6QMOZL5GPZAMPQ5P2BVSD6HUDQ, seg_pos={0 0}, upload=832686]: Done uploading piece. err: ecclient: upload failed (node:12ajyozgSuf68kn8sUuEmuSfYCcxzU8Cwmg1EXVt9WV6dKSopy7, address:45.91.116.13:4166): protocol: write tcp 159.223.143.39:45960->45.91.116.13:4166: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 6 10:40:12.258934014 (743h40m9.855705039s) [seg_pos={0 0}, upload=832686]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 6 10:40:12.278983295 (743h40m9.875754318s) [upload=832686]: Done uploading segments... Apr 6 10:40:12.279076909 (743h40m9.875847939s) [upload=832686]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.2735797s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 683.543456ms trace: https://jaeger.dev.storj.io/trace/7756830e434d8498 FAIL (2026-04-06 11:19:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 683.638227ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [......XXX...................................................] duration: 644.542718ms trace: https://jaeger.dev.storj.io/trace/4736d76b51c361a3 FAIL (2026-04-06 11:19:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 644.574263ms core uplink us-select-1 us1-placement-20

1
...
13
14
15
16
17
...
6260