Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 1.191848048s trace: https://jaeger.dev.storj.io/trace/15ce03727facea57 FAIL (2026-03-18 19:38:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Mar 18 19:36:10.500432381 (296h36m8.097203414s) [upload=332219]: Starting upload Mar 18 19:36:10.500543799 (296h36m8.097314825s) [upload=332219]: Done starting upload Mar 18 19:36:10.500649790 (296h36m8.097420808s) [upload=332219]: Uploading segments... Mar 18 19:36:10.500874959 (296h36m8.097645973s) [upload=332219]: Got next segment. Inline: false Mar 18 19:36:10.500932640 (296h36m8.097703652s) [upload=332219]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 19:36:10.500938749 (296h36m8.097709772s) [upload=332219]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 19:36:10.530892350 (296h36m8.127663383s) [seg_pos={0 0}, upload=332219]: Begin upload segment... Mar 18 19:36:10.530984837 (296h36m8.127755871s) [seg_pos={0 0}, upload=332219]: Done begin upload segment. Mar 18 19:36:10.531072762 (296h36m8.127843796s) [noise=true , address=145.79.227.52:4458 , piece_id=SH4VV6STCFPDZ4QQUKDYJKQIH6Z7X4HPFEGG4UHLAMTX7U54K53Q, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.531740352 (296h36m8.128511384s) [noise=true , address=79.127.205.235:4272 , piece_id=4LY4OR3XG7TPI63P2Q5KVLX5LMU3XKBH4C3SMD3SNZRW3EGGJHAA, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.531870513 (296h36m8.128641539s) [noise=true , address=79.127.219.38:4851 , piece_id=HAAHZMKR5BH5JBKIM2C73226FZRBVEZESYDQATE6ZJNKHH4ZPCDQ, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.532746824 (296h36m8.129517851s) [noise=true , address=77.90.31.21:4903 , piece_id=JBH363RHO4FJ2G4HJIDRQM4U3H6BW2ON7DLYZLDL6WUEWKGCNZCQ, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.532993109 (296h36m8.129764133s) [noise=true , address=77.90.31.17:4798 , piece_id=PRUFHXZXFJSV456IJM4YGC6FO65S4W24K5QGXWSAJZC4H77MWYBQ, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.533827706 (296h36m8.130598738s) [noise=true , address=145.79.227.3:4156 , piece_id=E67ZTTSMDV2WBUAI4O7RUQ2QWHMA7VSGRSF32NKD4AZRP6FXKLVA, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.533923098 (296h36m8.130694124s) [noise=true , address=79.127.219.46:4919 , piece_id=CWWJPNSKGWZRN5JUBINAAV2TFLDTBH7DZRMV6V7SV7ZBAWROYNKQ, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.534804584 (296h36m8.131575613s) [noise=true , address=145.79.227.36:4540 , piece_id=C6JUTP5TWCMIIHQQOPCXNDTNNCXN5Y37IZQAIK6O6RHELC3ZO6AA, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.534867539 (296h36m8.131638564s) [noise=true , address=193.149.164.70:4106 , piece_id=RBCC4FVQZBNI74A4SCXY2OFDBBNMYO5FH32CPHPZDN3XOG2KEJLQ, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.535728720 (296h36m8.132499754s) [noise=true , address=145.79.227.22:4538 , piece_id=Y2MRTJ6FUF7IY6TA3RQKMTHHDJ6E62JUZ5I7K7J5D5BRR2HPPH7A, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.535783456 (296h36m8.132554487s) [noise=true , address=193.149.164.69:4081 , piece_id=JORHOCGDU6SNE57F4WVH4AUVJ3HAX3V6MSB7L4YNBWKLXRC7K5LQ, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.536539101 (296h36m8.133310129s) [noise=true , address=193.149.164.68:4060 , piece_id=XY5RJPNM5BM27A6MJDK26WSTBHC3RJBZSRXS3VS32YUT2M7Q6UPA, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.536922919 (296h36m8.133693949s) [noise=true , address=145.79.227.25:4635 , piece_id=T7LMGZK2TPA5GXWJZG3OYECOPFIY2EXLYIISGOFMVX7HLUCYHPTQ, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.537283766 (296h36m8.134054813s) [noise=true , address=79.127.219.45:4868 , piece_id=LU76MTXKI7UVDV5VHXLFGZO3CLWKEQJGR463E3J2LDLDB57AA3ZQ, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.538845517 (296h36m8.135616541s) [noise=true , address=77.90.31.16:4752 , piece_id=MH7M34YWITMHOLYDHZAA5HZOVME6A6NSZDUALQJRPQNAMP22PMAQ, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.539332130 (296h36m8.136103158s) [noise=true , address=167.17.72.69:4719 , piece_id=2AY2C35O5IKGH45UWFYDHSA6QKSCQVVIK3WCNKTNOWCCKGWQY5WA, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.539744551 (296h36m8.136515577s) [noise=true , address=167.17.72.68:4661 , piece_id=VWCENPL2O42UCC6JOUFH337C2DDY3CYNTPCKPNGFPRWLIYE7YUCA, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.540083638 (296h36m8.136854667s) [noise=true , address=45.91.116.14:4209 , piece_id=NUOPZWUJ7QTBQ3RWNRYC3H4I2DACNCYZ75XQYOBLUSITNU2PK3LQ, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.601600388 (296h36m8.198371421s) [upload=332219]: Next returned nil segment Mar 18 19:36:10.601617944 (296h36m8.198388973s) [upload=332219]: Waiting for error group managing segments... Mar 18 19:36:10.606706422 (296h36m8.20347745s) [noise=true , address=77.90.31.16:4752 , piece_id=MH7M34YWITMHOLYDHZAA5HZOVME6A6NSZDUALQJRPQNAMP22PMAQ, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:10.607550275 (296h36m8.204321303s) [noise=true , address=79.127.219.45:4868 , piece_id=LU76MTXKI7UVDV5VHXLFGZO3CLWKEQJGR463E3J2LDLDB57AA3ZQ, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:10.608806703 (296h36m8.205577728s) [noise=true , address=79.127.219.38:4851 , piece_id=HAAHZMKR5BH5JBKIM2C73226FZRBVEZESYDQATE6ZJNKHH4ZPCDQ, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:10.622912404 (296h36m8.219683443s) [noise=true , address=45.91.116.14:4209 , piece_id=NUOPZWUJ7QTBQ3RWNRYC3H4I2DACNCYZ75XQYOBLUSITNU2PK3LQ, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:10.623232794 (296h36m8.220003819s) [noise=true , address=77.90.31.17:4798 , piece_id=PRUFHXZXFJSV456IJM4YGC6FO65S4W24K5QGXWSAJZC4H77MWYBQ, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:10.623919545 (296h36m8.220690574s) [noise=true , address=79.127.219.46:4919 , piece_id=CWWJPNSKGWZRN5JUBINAAV2TFLDTBH7DZRMV6V7SV7ZBAWROYNKQ, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:10.669238480 (296h36m8.266009504s) [noise=true , address=77.90.31.21:4903 , piece_id=JBH363RHO4FJ2G4HJIDRQM4U3H6BW2ON7DLYZLDL6WUEWKGCNZCQ, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:10.849192400 (296h36m8.44596344s) [noise=true , address=167.17.72.68:4661 , piece_id=VWCENPL2O42UCC6JOUFH337C2DDY3CYNTPCKPNGFPRWLIYE7YUCA, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:10.899652863 (296h36m8.496423891s) [noise=true , address=167.17.72.69:4719 , piece_id=2AY2C35O5IKGH45UWFYDHSA6QKSCQVVIK3WCNKTNOWCCKGWQY5WA, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:11.118324074 (296h36m8.715095097s) [noise=true , address=79.127.205.235:4272 , piece_id=4LY4OR3XG7TPI63P2Q5KVLX5LMU3XKBH4C3SMD3SNZRW3EGGJHAA, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:11.127553350 (296h36m8.724324364s) [noise=true , address=193.149.164.69:4081 , piece_id=JORHOCGDU6SNE57F4WVH4AUVJ3HAX3V6MSB7L4YNBWKLXRC7K5LQ, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:11.145684011 (296h36m8.742455034s) [noise=true , address=193.149.164.68:4060 , piece_id=XY5RJPNM5BM27A6MJDK26WSTBHC3RJBZSRXS3VS32YUT2M7Q6UPA, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:11.146892159 (296h36m8.743663183s) [noise=true , address=193.149.164.70:4106 , piece_id=RBCC4FVQZBNI74A4SCXY2OFDBBNMYO5FH32CPHPZDN3XOG2KEJLQ, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:37:10.457013911 (296h37m8.05378494s) [noise=true , address=145.79.227.52:4458 , piece_id=SH4VV6STCFPDZ4QQUKDYJKQIH6Z7X4HPFEGG4UHLAMTX7U54K53Q, seg_pos={0 0}, upload=332219]: Done uploading piece. err: ecclient: upload failed (node:1TMPdLfCbmgi3GqDDg6AUvTCFPzbCCgwTAn9JJBrYJRnbkKtpb, address:145.79.227.52:4458): protocol: rpc: dial tcp 145.79.227.52:4458: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:10.457080518 (296h37m8.053851548s) [noise=true , address=145.79.227.3:4156 , piece_id=E67ZTTSMDV2WBUAI4O7RUQ2QWHMA7VSGRSF32NKD4AZRP6FXKLVA, seg_pos={0 0}, upload=332219]: Done uploading piece. err: ecclient: upload failed (node:12JTrtKCNHFUY91Ktaa7v7eAWe4GHssKSeofrg3diu8isHJnmy4, address:145.79.227.3:4156): protocol: rpc: dial tcp 145.79.227.3:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:10.457355833 (296h37m8.054126865s) [noise=true , address=145.79.227.36:4540 , piece_id=C6JUTP5TWCMIIHQQOPCXNDTNNCXN5Y37IZQAIK6O6RHELC3ZO6AA, seg_pos={0 0}, upload=332219]: Done uploading piece. err: ecclient: upload failed (node:1P92CRZqnx1avdDo18tMY2X7FujkZugb4gU98AaErD13gX4cLA, address:145.79.227.36:4540): protocol: rpc: dial tcp 145.79.227.36:4540: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:10.458754175 (296h37m8.055525209s) [noise=true , address=145.79.227.22:4538 , piece_id=Y2MRTJ6FUF7IY6TA3RQKMTHHDJ6E62JUZ5I7K7J5D5BRR2HPPH7A, seg_pos={0 0}, upload=332219]: Done uploading piece. err: ecclient: upload failed (node:1kNFzPpkDtwy4VFyzeFQReorJPEVsmdDAd8ULT8ZKoGde57P3v, address:145.79.227.22:4538): protocol: rpc: dial tcp 145.79.227.22:4538: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:10.459003879 (296h37m8.055774913s) [noise=true , address=145.79.227.25:4635 , piece_id=T7LMGZK2TPA5GXWJZG3OYECOPFIY2EXLYIISGOFMVX7HLUCYHPTQ, seg_pos={0 0}, upload=332219]: Done uploading piece. err: ecclient: upload failed (node:12jyQ3G4H8fYWdP6RiREP93PzZq3G9wm2BKKk2tZSfcmyopP1a2, address:145.79.227.25:4635): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:37:10.459074189 (296h37m8.055845199s) [seg_pos={0 0}, upload=332219]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Mar 18 19:37:10.482165416 (296h37m8.078936449s) [upload=332219]: Done uploading segments... Mar 18 19:37:10.482224140 (296h37m8.078995174s) [upload=332219]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.191867606s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 9 out of 60 is failed (threshold=6) [...................................................XXXXXXXXX] duration: 5.001477139s trace: https://jaeger.dev.storj.io/trace/238c986c2668b78e FAIL (2026-03-18 19:38:10 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001577672s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................................................XXX..] duration: 1.284819915s trace: https://jaeger.dev.storj.io/trace/504da329c61bc56 FAIL (2026-03-18 19:37:57 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== in upload segment... Mar 18 19:36:40.519062491 (296h36m38.115833524s) [seg_pos={0 0}, upload=332226]: Done begin upload segment. Mar 18 19:36:40.519122972 (296h36m38.115893995s) [noise=true , address=167.17.72.70:4756 , piece_id=RF4MNRT5B5WSDQ5OE4YIDS6FJJRKMGWEEDMPZU4GUESEDQZUYS4A, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.519158535 (296h36m38.115929568s) [noise=true , address=193.149.164.70:4137 , piece_id=IK2NPZKZECM55AGMLYHMGRML4RT677DLAQK2RUPNLTS2DPWZESJA, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.519952691 (296h36m38.116723717s) [noise=true , address=145.79.227.39:4045 , piece_id=ONYBRZMCR6WZIAFOJ43XMTLVGBCH7GYTGBT5NBSY6U26WPOR72AQ, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.520469473 (296h36m38.117240506s) [noise=true , address=77.90.31.17:4783 , piece_id=K7YEUHBNATYTZ3AGF3PJB27L6SFI5J5PLDO3F6WTIHW45WBBOMGQ, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.520531096 (296h36m38.117302138s) [noise=true , address=77.90.31.13:4640 , piece_id=JWLWW3WC5WL3W7C774PU5EFQJJVRHGRDXTIJLIWQFLMBSV4BFSVQ, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.521025199 (296h36m38.117796226s) [noise=true , address=145.79.227.2:4135 , piece_id=QS2HTEINCAU5IRXJHWJPZPOUXUER7LV2LA7GLPRMLXZKZM5PRPCQ, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.521195168 (296h36m38.117966191s) [noise=true , address=145.79.227.27:4701 , piece_id=GIZEBKJF5M7QBMLYPN3WQUIJYVHHD6HXEVPGKAIIOLA273AWTPPQ, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.521517234 (296h36m38.118288257s) [noise=true , address=145.79.227.37:4589 , piece_id=Z3ZPG2RWAH375JWNP3Q42MJHOG4AVZBQOAS27ISLSB7TOVFUSNHQ, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.521721504 (296h36m38.118492529s) [noise=true , address=64.49.13.137:4503 , piece_id=CGB3TCGOS6Y3UHUJS7MLV24FUDN3JVHPXQ2QIIETRONZ4H7C6S3A, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.522279589 (296h36m38.119050616s) [noise=true , address=79.127.219.46:4911 , piece_id=RGSZ7VYJJCBNEPJZ6FJ7BM6BQGOSBZQLRWXHNNB4C2C3PSN43NNA, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.523062176 (296h36m38.119833208s) [noise=true , address=77.90.31.12:4595 , piece_id=ZTJWMLII2CK4LHI3TXZWZIADFIJQAD2E5ID44UEUQFRGMG3EU54A, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.523708767 (296h36m38.12047979s) [noise=true , address=145.79.227.36:4540 , piece_id=NCP3622NO7K3ZHBNOZVWRRCRHIQKHW5RN7N5OQCLL2UPSODMA4ZQ, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.524314790 (296h36m38.121085826s) [noise=true , address=145.79.227.40:4062 , piece_id=BXDV657N73AWJLF54QJWYQ5Q6KTO62AKXVYVRCYKPXYNGYDH6R4A, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.524789568 (296h36m38.121560591s) [noise=true , address=193.149.164.67:4008 , piece_id=Y6NJEIGE5JWOBWVMTQO4BYBSY2BX7OE45TKJI3BVX7KKQYMR3Z5Q, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.525324320 (296h36m38.122095333s) [noise=true , address=79.127.205.234:4206 , piece_id=233HW3UMHJLRTISXVCO2PR3UREECB63KELEBX4ZLYGZ7JPGRGQLA, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.525779446 (296h36m38.122550472s) [noise=true , address=79.127.219.45:4876 , piece_id=3TTZKRJ5BQF6IMQ6SRTH7DK2ZX7LXJLDPYDLIBIERU4WSRCYIWFQ, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.526254332 (296h36m38.123025355s) [noise=true , address=167.17.72.71:4784 , piece_id=ZBUUOXLEZQ353DQOY3ZSKI22H4T6VPHCDFMTQPMLGZBHYUYUXIBA, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.526725151 (296h36m38.123496178s) [noise=true , address=145.79.227.52:4475 , piece_id=QFVWGT7WIWXIHX3OXZKXWRZ3HNKF4EXN7C6U2OFSQMZF56QCCHQA, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.589597769 (296h36m38.186368792s) [upload=332226]: Next returned nil segment Mar 18 19:36:40.589617873 (296h36m38.186388903s) [upload=332226]: Waiting for error group managing segments... Mar 18 19:36:40.594936043 (296h36m38.191707073s) [noise=true , address=77.90.31.12:4595 , piece_id=ZTJWMLII2CK4LHI3TXZWZIADFIJQAD2E5ID44UEUQFRGMG3EU54A, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:36:40.597084865 (296h36m38.193855887s) [noise=true , address=79.127.219.45:4876 , piece_id=3TTZKRJ5BQF6IMQ6SRTH7DK2ZX7LXJLDPYDLIBIERU4WSRCYIWFQ, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:36:40.610365162 (296h36m38.207136186s) [noise=true , address=77.90.31.13:4640 , piece_id=JWLWW3WC5WL3W7C774PU5EFQJJVRHGRDXTIJLIWQFLMBSV4BFSVQ, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:36:40.623540288 (296h36m38.220311316s) [noise=true , address=77.90.31.17:4783 , piece_id=K7YEUHBNATYTZ3AGF3PJB27L6SFI5J5PLDO3F6WTIHW45WBBOMGQ, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:36:40.637765186 (296h36m38.234536209s) [noise=true , address=79.127.219.46:4911 , piece_id=RGSZ7VYJJCBNEPJZ6FJ7BM6BQGOSBZQLRWXHNNB4C2C3PSN43NNA, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:36:40.849102006 (296h36m38.445873019s) [noise=true , address=167.17.72.70:4756 , piece_id=RF4MNRT5B5WSDQ5OE4YIDS6FJJRKMGWEEDMPZU4GUESEDQZUYS4A, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:36:40.854370150 (296h36m38.451141177s) [noise=true , address=167.17.72.71:4784 , piece_id=ZBUUOXLEZQ353DQOY3ZSKI22H4T6VPHCDFMTQPMLGZBHYUYUXIBA, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:36:41.113067742 (296h36m38.709838765s) [noise=true , address=193.149.164.67:4008 , piece_id=Y6NJEIGE5JWOBWVMTQO4BYBSY2BX7OE45TKJI3BVX7KKQYMR3Z5Q, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:36:41.117252653 (296h36m38.714023687s) [noise=true , address=79.127.205.234:4206 , piece_id=233HW3UMHJLRTISXVCO2PR3UREECB63KELEBX4ZLYGZ7JPGRGQLA, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:36:41.118717458 (296h36m38.715488481s) [noise=true , address=64.49.13.137:4503 , piece_id=CGB3TCGOS6Y3UHUJS7MLV24FUDN3JVHPXQ2QIIETRONZ4H7C6S3A, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:36:41.123965737 (296h36m38.720736764s) [noise=true , address=193.149.164.70:4137 , piece_id=IK2NPZKZECM55AGMLYHMGRML4RT677DLAQK2RUPNLTS2DPWZESJA, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:37:40.459260224 (296h37m38.056031257s) [noise=true , address=145.79.227.37:4589 , piece_id=Z3ZPG2RWAH375JWNP3Q42MJHOG4AVZBQOAS27ISLSB7TOVFUSNHQ, seg_pos={0 0}, upload=332226]: Done uploading piece. err: ecclient: upload failed (node:12ChutvH6jC7eWE7dDE9Cmsn8EDo6ecHum8dG9NBvEcdphiJgbi, address:145.79.227.37:4589): protocol: rpc: dial tcp 145.79.227.37:4589: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:40.459832101 (296h37m38.056603126s) [noise=true , address=145.79.227.52:4475 , piece_id=QFVWGT7WIWXIHX3OXZKXWRZ3HNKF4EXN7C6U2OFSQMZF56QCCHQA, seg_pos={0 0}, upload=332226]: Done uploading piece. err: ecclient: upload failed (node:12UYjE6PEB9gZKwTKcnx23ykF4jDMJzLQoKmiFiejWew5mirdCj, address:145.79.227.52:4475): protocol: rpc: dial tcp 145.79.227.52:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:40.460091051 (296h37m38.056862082s) [noise=true , address=145.79.227.2:4135 , piece_id=QS2HTEINCAU5IRXJHWJPZPOUXUER7LV2LA7GLPRMLXZKZM5PRPCQ, seg_pos={0 0}, upload=332226]: Done uploading piece. err: ecclient: upload failed (node:1uEBWRaMH4mLApUcgHyHQmGxxbyXnYUScg5Z2on9LHESYkqj3S, address:145.79.227.2:4135): protocol: rpc: dial tcp 145.79.227.2:4135: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:40.460212199 (296h37m38.056983231s) [noise=true , address=145.79.227.39:4045 , piece_id=ONYBRZMCR6WZIAFOJ43XMTLVGBCH7GYTGBT5NBSY6U26WPOR72AQ, seg_pos={0 0}, upload=332226]: Done uploading piece. err: ecclient: upload failed (node:1j6J2T4aX3bmjtP9J379mNQBFDp3vQZeAvmUFEYVfYZFknjrTp, address:145.79.227.39:4045): protocol: rpc: dial tcp 145.79.227.39:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:40.460371799 (296h37m38.05714283s) [noise=true , address=145.79.227.27:4701 , piece_id=GIZEBKJF5M7QBMLYPN3WQUIJYVHHD6HXEVPGKAIIOLA273AWTPPQ, seg_pos={0 0}, upload=332226]: Done uploading piece. err: ecclient: upload failed (node:1weCZWAtZRRYdeuPE8TjnZB5fZVD4gz4qW6TxyyNz33wVfW9hQ, address:145.79.227.27:4701): protocol: rpc: dial tcp 145.79.227.27:4701: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:40.460512619 (296h37m38.057283649s) [noise=true , address=145.79.227.40:4062 , piece_id=BXDV657N73AWJLF54QJWYQ5Q6KTO62AKXVYVRCYKPXYNGYDH6R4A, seg_pos={0 0}, upload=332226]: Done uploading piece. err: ecclient: upload failed (node:12N61Va6p4WU9cmUz65CkCDKr54LFtJhh8HJGYPVt8cJTRoWLhb, address:145.79.227.40:4062): protocol: rpc: dial tcp 145.79.227.40:4062: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:40.460673235 (296h37m38.057444267s) [noise=true , address=145.79.227.36:4540 , piece_id=NCP3622NO7K3ZHBNOZVWRRCRHIQKHW5RN7N5OQCLL2UPSODMA4ZQ, seg_pos={0 0}, upload=332226]: Done uploading piece. err: ecclient: upload failed (node:1P92CRZqnx1avdDo18tMY2X7FujkZugb4gU98AaErD13gX4cLA, address:145.79.227.36:4540): protocol: rpc: dial tcp 145.79.227.36:4540: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:40.460706522 (296h37m38.057477535s) [seg_pos={0 0}, upload=332226]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Mar 18 19:37:40.478800721 (296h37m38.075571752s) [upload=332226]: Done uploading segments... Mar 18 19:37:40.478897894 (296h37m38.075668927s) [upload=332226]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 1.284852362s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [........................................................XXXX] duration: 1m0.000742829s trace: https://jaeger.dev.storj.io/trace/77badf6a4d45c593 FAIL (2026-03-18 19:37:51 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000778737s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [....................................................XXXXXXXX] duration: 5.00068565s trace: https://jaeger.dev.storj.io/trace/19a33af9bc6a7441 FAIL (2026-03-18 19:37:45 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000709266s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [......................................................XXXXXX] duration: 30.022222967s trace: https://jaeger.dev.storj.io/trace/223554fe0488ccea FAIL (2026-03-18 19:37:42 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== Mar 18 19:37:12.297729750 (296h37m9.894500784s) [noise=true , address=45.91.116.14:4192 , piece_id=T4KDR3VYB37SX2IBN767YOSYCL2GCDBJH4TAM4BKE2KIU7M3JNYQ, seg_pos={0 0}, upload=332239]: Uploading piece... Mar 18 19:37:12.298829822 (296h37m9.895600884s) [noise=true , address=145.79.227.19:4080 , piece_id=MBGPIBD4YZB7O57HLIB2UVBXJKEWRLH5STPQ3TFKYT7DF5T7UA2Q, seg_pos={0 0}, upload=332239]: Uploading piece... Mar 18 19:37:12.299135287 (296h37m9.895906317s) [seg_pos={0 0}, upload=332239]: Done begin upload segment. Mar 18 19:37:12.299191925 (296h37m9.895962959s) [noise=true , address=45.91.116.7:4012 , piece_id=5MVMV6VFDL44GY4YN7UKU42CT2ZFOUI5CIAEDN5Y6MML34BGAWAA, seg_pos={0 0}, upload=332239]: Uploading piece... Mar 18 19:37:12.299760194 (296h37m9.896531228s) [noise=true , address=145.79.227.52:4447 , piece_id=3CILEPLT3JHAJILM3E4JPH7QEAZXBIFFTZEPILVVXLVO5VUXNNAQ, seg_pos={0 0}, upload=332239]: Uploading piece... Mar 18 19:37:12.299807221 (296h37m9.896578258s) [noise=true , address=145.79.227.18:4008 , piece_id=XQNAXXTOQFBH4BLVLDG2WRG4KG676OS6IUELHDY6EFKYAXVINSJQ, seg_pos={0 0}, upload=332239]: Uploading piece... Mar 18 19:37:12.300271145 (296h37m9.897042175s) [noise=true , address=145.79.227.43:4157 , piece_id=XM6GV6NEK4S3GITPCRJ5OWZH3K3R57OYF4DTVBNRSH6TYK3U2DVQ, seg_pos={0 0}, upload=332239]: Uploading piece... Mar 18 19:37:12.300878550 (296h37m9.897649583s) [noise=true , address=145.79.227.44:4194 , piece_id=QZLQIRQ2WQWB337724YGNA7STPBJY4XYWXD65REZXOQIRCFUY5VA, seg_pos={0 0}, upload=332239]: Uploading piece... Mar 18 19:37:12.301367078 (296h37m9.898138112s) [noise=true , address=145.79.227.28:4731 , piece_id=2BZHWEUEC7QRIZH637ADEPAI3SRWNXSUAGXRYR6O2MMDBJP6NU5Q, seg_pos={0 0}, upload=332239]: Uploading piece... Mar 18 19:37:12.301870621 (296h37m9.89864165s) [noise=true , address=79.127.219.45:4853 , piece_id=265XNOV5B5BPRMHB6Y6OG4CM62D7L76QKF5VK4LFCR22CHNSOA2A, seg_pos={0 0}, upload=332239]: Uploading piece... Mar 18 19:37:12.302405047 (296h37m9.89917608s) [noise=true , address=145.79.227.2:4119 , piece_id=2XGCEDTYKELPX7S5YE7KHD3OLUMGFGFVDZKL5EZ5QG4QBP7ZFZZA, seg_pos={0 0}, upload=332239]: Uploading piece... Mar 18 19:37:12.302520459 (296h37m9.899291493s) [noise=true , address=45.91.116.10:4083 , piece_id=PQR75LP3MAROJ554KWZYEPXDNABGIXWAJFRNXKKPFD764DNBC36Q, seg_pos={0 0}, upload=332239]: Uploading piece... Mar 18 19:37:12.303005751 (296h37m9.899776794s) [noise=true , address=45.91.116.12:4244 , piece_id=YX3JVIKMLQNJXSIO5FAH2MG5OVIKHUUMBJLZKKVAQ7IWBTZ4NRMA, seg_pos={0 0}, upload=332239]: Uploading piece... Mar 18 19:37:12.303636584 (296h37m9.900407611s) [noise=true , address=145.79.227.20:4441 , piece_id=7T6AFGZ4WBMGZ4AYIXXL5C4RU4UP3ADNT7FFWFSOJ2MNINZ4RFNA, seg_pos={0 0}, upload=332239]: Uploading piece... Mar 18 19:37:12.304248026 (296h37m9.901019057s) [noise=true , address=145.79.227.46:4606 , piece_id=OEWHVCDAIZXIN5XKP5J2SELOQPLWKO3E77OHE2L6S2AE2QENENCA, seg_pos={0 0}, upload=332239]: Uploading piece... Mar 18 19:37:12.305130692 (296h37m9.901901728s) [noise=true , address=145.79.227.37:4566 , piece_id=XXFKTQRX6EAVPSR43VJF2W7TWPSEDLA76QUHJ73EEEYLHX7RWOZA, seg_pos={0 0}, upload=332239]: Uploading piece... Mar 18 19:37:12.305821964 (296h37m9.902592997s) [noise=true , address=145.79.227.31:4800 , piece_id=AMF7KVGIEZYRURTC6ZZGRFZHV4Q4RNIHJG3JNXTTFQGVK2BB5W7A, seg_pos={0 0}, upload=332239]: Uploading piece... Mar 18 19:37:12.382654341 (296h37m9.979425358s) [upload=332239]: Next returned nil segment Mar 18 19:37:12.382669066 (296h37m9.979440092s) [upload=332239]: Waiting for error group managing segments... Mar 18 19:37:12.386847040 (296h37m9.983618052s) [noise=true , address=79.127.219.45:4853 , piece_id=265XNOV5B5BPRMHB6Y6OG4CM62D7L76QKF5VK4LFCR22CHNSOA2A, seg_pos={0 0}, upload=332239]: Done uploading piece. err: <nil> Mar 18 19:37:12.388036835 (296h37m9.984807852s) [noise=true , address=45.91.116.12:4244 , piece_id=YX3JVIKMLQNJXSIO5FAH2MG5OVIKHUUMBJLZKKVAQ7IWBTZ4NRMA, seg_pos={0 0}, upload=332239]: Done uploading piece. err: <nil> Mar 18 19:37:12.388065515 (296h37m9.984836527s) [noise=true , address=45.91.116.10:4083 , piece_id=PQR75LP3MAROJ554KWZYEPXDNABGIXWAJFRNXKKPFD764DNBC36Q, seg_pos={0 0}, upload=332239]: Done uploading piece. err: <nil> Mar 18 19:37:12.388573437 (296h37m9.98534445s) [noise=true , address=45.91.116.14:4192 , piece_id=T4KDR3VYB37SX2IBN767YOSYCL2GCDBJH4TAM4BKE2KIU7M3JNYQ, seg_pos={0 0}, upload=332239]: Done uploading piece. err: <nil> Mar 18 19:37:12.410081945 (296h37m10.006852959s) [noise=true , address=45.91.116.7:4012 , piece_id=5MVMV6VFDL44GY4YN7UKU42CT2ZFOUI5CIAEDN5Y6MML34BGAWAA, seg_pos={0 0}, upload=332239]: Done uploading piece. err: <nil> Mar 18 19:37:42.236885070 (296h37m39.833656103s) [noise=true , address=145.79.227.18:4008 , piece_id=XQNAXXTOQFBH4BLVLDG2WRG4KG676OS6IUELHDY6EFKYAXVINSJQ, seg_pos={0 0}, upload=332239]: Done uploading piece. err: ecclient: upload failed (node:148CqQmrsJSfoGByHYRPoZmvP1VgQQpaMVLb7RgRQvNBuo7air, address:145.79.227.18:4008): protocol: rpc: dial tcp 145.79.227.18:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:42.236957861 (296h37m39.833728896s) [noise=true , address=145.79.227.31:4800 , piece_id=AMF7KVGIEZYRURTC6ZZGRFZHV4Q4RNIHJG3JNXTTFQGVK2BB5W7A, seg_pos={0 0}, upload=332239]: Done uploading piece. err: ecclient: upload failed (node:121yHupysREmASSVx3WUN57duu16QEs7sA2BPBWyaThFzruvch2, address:145.79.227.31:4800): protocol: rpc: dial tcp 145.79.227.31:4800: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:42.237233623 (296h37m39.834004657s) [noise=true , address=145.79.227.2:4119 , piece_id=2XGCEDTYKELPX7S5YE7KHD3OLUMGFGFVDZKL5EZ5QG4QBP7ZFZZA, seg_pos={0 0}, upload=332239]: Done uploading piece. err: ecclient: upload failed (node:121xNAnkm9HZXUszp26UmWTsuhp3igyeDVea6gSCdtrQUQDVHpW, address:145.79.227.2:4119): protocol: rpc: dial tcp 145.79.227.2:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:42.237411887 (296h37m39.83418292s) [noise=true , address=145.79.227.44:4194 , piece_id=QZLQIRQ2WQWB337724YGNA7STPBJY4XYWXD65REZXOQIRCFUY5VA, seg_pos={0 0}, upload=332239]: Done uploading piece. err: ecclient: upload failed (node:12kta1F6RNDP9GJdkiUG3x1Nk6juWJ2tmSDr599DYomSq9C1BBh, address:145.79.227.44:4194): protocol: rpc: dial tcp 145.79.227.44:4194: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:42.237569844 (296h37m39.834340877s) [noise=true , address=145.79.227.37:4566 , piece_id=XXFKTQRX6EAVPSR43VJF2W7TWPSEDLA76QUHJ73EEEYLHX7RWOZA, seg_pos={0 0}, upload=332239]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: rpc: dial tcp 145.79.227.37:4566: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:42.237755725 (296h37m39.83452676s) [noise=true , address=145.79.227.19:4080 , piece_id=MBGPIBD4YZB7O57HLIB2UVBXJKEWRLH5STPQ3TFKYT7DF5T7UA2Q, seg_pos={0 0}, upload=332239]: Done uploading piece. err: ecclient: upload failed (node:12KK51wudKiZb39Ku7wZhgBYEYmgsZTHfjiNNLmpBU9uuNFapt3, address:145.79.227.19:4080): protocol: rpc: dial tcp 145.79.227.19:4080: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:42.237949934 (296h37m39.834720967s) [noise=true , address=145.79.227.52:4447 , piece_id=3CILEPLT3JHAJILM3E4JPH7QEAZXBIFFTZEPILVVXLVO5VUXNNAQ, seg_pos={0 0}, upload=332239]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:42.238010768 (296h37m39.834781801s) [noise=true , address=145.79.227.46:4606 , piece_id=OEWHVCDAIZXIN5XKP5J2SELOQPLWKO3E77OHE2L6S2AE2QENENCA, seg_pos={0 0}, upload=332239]: Done uploading piece. err: ecclient: upload failed (node:12SMRAQn7BaX1Fzpvou2JHaiLko9fcQLdYggaVcjRihU516CbRh, address:145.79.227.46:4606): protocol: rpc: dial tcp 145.79.227.46:4606: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:42.238233107 (296h37m39.835004141s) [noise=true , address=145.79.227.43:4157 , piece_id=XM6GV6NEK4S3GITPCRJ5OWZH3K3R57OYF4DTVBNRSH6TYK3U2DVQ, seg_pos={0 0}, upload=332239]: Done uploading piece. err: ecclient: upload failed (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz, address:145.79.227.43:4157): protocol: rpc: dial tcp 145.79.227.43:4157: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:42.238581925 (296h37m39.83535296s) [noise=true , address=145.79.227.28:4731 , piece_id=2BZHWEUEC7QRIZH637ADEPAI3SRWNXSUAGXRYR6O2MMDBJP6NU5Q, seg_pos={0 0}, upload=332239]: Done uploading piece. err: ecclient: upload failed (node:12NQzTwVYQa6ZzxAHQm1apkwWnqJstTnAJQyHKhBzC2jTN9kEVK, address:145.79.227.28:4731): protocol: rpc: dial tcp 145.79.227.28:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:42.238837019 (296h37m39.835608056s) [noise=true , address=145.79.227.20:4441 , piece_id=7T6AFGZ4WBMGZ4AYIXXL5C4RU4UP3ADNT7FFWFSOJ2MNINZ4RFNA, seg_pos={0 0}, upload=332239]: Done uploading piece. err: ecclient: upload failed (node:12LjGecRNJE1kq67iMtxZFhWeaUZpUogsGxbz4VXwnHdQtTnbBr, address:145.79.227.20:4441): protocol: rpc: dial tcp 145.79.227.20:4441: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:42.238884580 (296h37m39.835655604s) [seg_pos={0 0}, upload=332239]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Mar 18 19:37:42.257146928 (296h37m39.853917969s) [upload=332239]: Done uploading segments... Mar 18 19:37:42.257186867 (296h37m39.853957901s) [upload=332239]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 30.022248835s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................................................XXX.] duration: 1.294218886s trace: https://jaeger.dev.storj.io/trace/504da329c61bc56 FAIL (2026-03-18 19:37:41 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== in upload segment... Mar 18 19:36:40.519062491 (296h36m38.115833524s) [seg_pos={0 0}, upload=332226]: Done begin upload segment. Mar 18 19:36:40.519122972 (296h36m38.115893995s) [noise=true , address=167.17.72.70:4756 , piece_id=RF4MNRT5B5WSDQ5OE4YIDS6FJJRKMGWEEDMPZU4GUESEDQZUYS4A, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.519158535 (296h36m38.115929568s) [noise=true , address=193.149.164.70:4137 , piece_id=IK2NPZKZECM55AGMLYHMGRML4RT677DLAQK2RUPNLTS2DPWZESJA, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.519952691 (296h36m38.116723717s) [noise=true , address=145.79.227.39:4045 , piece_id=ONYBRZMCR6WZIAFOJ43XMTLVGBCH7GYTGBT5NBSY6U26WPOR72AQ, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.520469473 (296h36m38.117240506s) [noise=true , address=77.90.31.17:4783 , piece_id=K7YEUHBNATYTZ3AGF3PJB27L6SFI5J5PLDO3F6WTIHW45WBBOMGQ, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.520531096 (296h36m38.117302138s) [noise=true , address=77.90.31.13:4640 , piece_id=JWLWW3WC5WL3W7C774PU5EFQJJVRHGRDXTIJLIWQFLMBSV4BFSVQ, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.521025199 (296h36m38.117796226s) [noise=true , address=145.79.227.2:4135 , piece_id=QS2HTEINCAU5IRXJHWJPZPOUXUER7LV2LA7GLPRMLXZKZM5PRPCQ, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.521195168 (296h36m38.117966191s) [noise=true , address=145.79.227.27:4701 , piece_id=GIZEBKJF5M7QBMLYPN3WQUIJYVHHD6HXEVPGKAIIOLA273AWTPPQ, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.521517234 (296h36m38.118288257s) [noise=true , address=145.79.227.37:4589 , piece_id=Z3ZPG2RWAH375JWNP3Q42MJHOG4AVZBQOAS27ISLSB7TOVFUSNHQ, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.521721504 (296h36m38.118492529s) [noise=true , address=64.49.13.137:4503 , piece_id=CGB3TCGOS6Y3UHUJS7MLV24FUDN3JVHPXQ2QIIETRONZ4H7C6S3A, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.522279589 (296h36m38.119050616s) [noise=true , address=79.127.219.46:4911 , piece_id=RGSZ7VYJJCBNEPJZ6FJ7BM6BQGOSBZQLRWXHNNB4C2C3PSN43NNA, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.523062176 (296h36m38.119833208s) [noise=true , address=77.90.31.12:4595 , piece_id=ZTJWMLII2CK4LHI3TXZWZIADFIJQAD2E5ID44UEUQFRGMG3EU54A, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.523708767 (296h36m38.12047979s) [noise=true , address=145.79.227.36:4540 , piece_id=NCP3622NO7K3ZHBNOZVWRRCRHIQKHW5RN7N5OQCLL2UPSODMA4ZQ, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.524314790 (296h36m38.121085826s) [noise=true , address=145.79.227.40:4062 , piece_id=BXDV657N73AWJLF54QJWYQ5Q6KTO62AKXVYVRCYKPXYNGYDH6R4A, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.524789568 (296h36m38.121560591s) [noise=true , address=193.149.164.67:4008 , piece_id=Y6NJEIGE5JWOBWVMTQO4BYBSY2BX7OE45TKJI3BVX7KKQYMR3Z5Q, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.525324320 (296h36m38.122095333s) [noise=true , address=79.127.205.234:4206 , piece_id=233HW3UMHJLRTISXVCO2PR3UREECB63KELEBX4ZLYGZ7JPGRGQLA, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.525779446 (296h36m38.122550472s) [noise=true , address=79.127.219.45:4876 , piece_id=3TTZKRJ5BQF6IMQ6SRTH7DK2ZX7LXJLDPYDLIBIERU4WSRCYIWFQ, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.526254332 (296h36m38.123025355s) [noise=true , address=167.17.72.71:4784 , piece_id=ZBUUOXLEZQ353DQOY3ZSKI22H4T6VPHCDFMTQPMLGZBHYUYUXIBA, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.526725151 (296h36m38.123496178s) [noise=true , address=145.79.227.52:4475 , piece_id=QFVWGT7WIWXIHX3OXZKXWRZ3HNKF4EXN7C6U2OFSQMZF56QCCHQA, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.589597769 (296h36m38.186368792s) [upload=332226]: Next returned nil segment Mar 18 19:36:40.589617873 (296h36m38.186388903s) [upload=332226]: Waiting for error group managing segments... Mar 18 19:36:40.594936043 (296h36m38.191707073s) [noise=true , address=77.90.31.12:4595 , piece_id=ZTJWMLII2CK4LHI3TXZWZIADFIJQAD2E5ID44UEUQFRGMG3EU54A, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:36:40.597084865 (296h36m38.193855887s) [noise=true , address=79.127.219.45:4876 , piece_id=3TTZKRJ5BQF6IMQ6SRTH7DK2ZX7LXJLDPYDLIBIERU4WSRCYIWFQ, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:36:40.610365162 (296h36m38.207136186s) [noise=true , address=77.90.31.13:4640 , piece_id=JWLWW3WC5WL3W7C774PU5EFQJJVRHGRDXTIJLIWQFLMBSV4BFSVQ, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:36:40.623540288 (296h36m38.220311316s) [noise=true , address=77.90.31.17:4783 , piece_id=K7YEUHBNATYTZ3AGF3PJB27L6SFI5J5PLDO3F6WTIHW45WBBOMGQ, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:36:40.637765186 (296h36m38.234536209s) [noise=true , address=79.127.219.46:4911 , piece_id=RGSZ7VYJJCBNEPJZ6FJ7BM6BQGOSBZQLRWXHNNB4C2C3PSN43NNA, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:36:40.849102006 (296h36m38.445873019s) [noise=true , address=167.17.72.70:4756 , piece_id=RF4MNRT5B5WSDQ5OE4YIDS6FJJRKMGWEEDMPZU4GUESEDQZUYS4A, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:36:40.854370150 (296h36m38.451141177s) [noise=true , address=167.17.72.71:4784 , piece_id=ZBUUOXLEZQ353DQOY3ZSKI22H4T6VPHCDFMTQPMLGZBHYUYUXIBA, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:36:41.113067742 (296h36m38.709838765s) [noise=true , address=193.149.164.67:4008 , piece_id=Y6NJEIGE5JWOBWVMTQO4BYBSY2BX7OE45TKJI3BVX7KKQYMR3Z5Q, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:36:41.117252653 (296h36m38.714023687s) [noise=true , address=79.127.205.234:4206 , piece_id=233HW3UMHJLRTISXVCO2PR3UREECB63KELEBX4ZLYGZ7JPGRGQLA, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:36:41.118717458 (296h36m38.715488481s) [noise=true , address=64.49.13.137:4503 , piece_id=CGB3TCGOS6Y3UHUJS7MLV24FUDN3JVHPXQ2QIIETRONZ4H7C6S3A, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:36:41.123965737 (296h36m38.720736764s) [noise=true , address=193.149.164.70:4137 , piece_id=IK2NPZKZECM55AGMLYHMGRML4RT677DLAQK2RUPNLTS2DPWZESJA, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:37:40.459260224 (296h37m38.056031257s) [noise=true , address=145.79.227.37:4589 , piece_id=Z3ZPG2RWAH375JWNP3Q42MJHOG4AVZBQOAS27ISLSB7TOVFUSNHQ, seg_pos={0 0}, upload=332226]: Done uploading piece. err: ecclient: upload failed (node:12ChutvH6jC7eWE7dDE9Cmsn8EDo6ecHum8dG9NBvEcdphiJgbi, address:145.79.227.37:4589): protocol: rpc: dial tcp 145.79.227.37:4589: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:40.459832101 (296h37m38.056603126s) [noise=true , address=145.79.227.52:4475 , piece_id=QFVWGT7WIWXIHX3OXZKXWRZ3HNKF4EXN7C6U2OFSQMZF56QCCHQA, seg_pos={0 0}, upload=332226]: Done uploading piece. err: ecclient: upload failed (node:12UYjE6PEB9gZKwTKcnx23ykF4jDMJzLQoKmiFiejWew5mirdCj, address:145.79.227.52:4475): protocol: rpc: dial tcp 145.79.227.52:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:40.460091051 (296h37m38.056862082s) [noise=true , address=145.79.227.2:4135 , piece_id=QS2HTEINCAU5IRXJHWJPZPOUXUER7LV2LA7GLPRMLXZKZM5PRPCQ, seg_pos={0 0}, upload=332226]: Done uploading piece. err: ecclient: upload failed (node:1uEBWRaMH4mLApUcgHyHQmGxxbyXnYUScg5Z2on9LHESYkqj3S, address:145.79.227.2:4135): protocol: rpc: dial tcp 145.79.227.2:4135: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:40.460212199 (296h37m38.056983231s) [noise=true , address=145.79.227.39:4045 , piece_id=ONYBRZMCR6WZIAFOJ43XMTLVGBCH7GYTGBT5NBSY6U26WPOR72AQ, seg_pos={0 0}, upload=332226]: Done uploading piece. err: ecclient: upload failed (node:1j6J2T4aX3bmjtP9J379mNQBFDp3vQZeAvmUFEYVfYZFknjrTp, address:145.79.227.39:4045): protocol: rpc: dial tcp 145.79.227.39:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:40.460371799 (296h37m38.05714283s) [noise=true , address=145.79.227.27:4701 , piece_id=GIZEBKJF5M7QBMLYPN3WQUIJYVHHD6HXEVPGKAIIOLA273AWTPPQ, seg_pos={0 0}, upload=332226]: Done uploading piece. err: ecclient: upload failed (node:1weCZWAtZRRYdeuPE8TjnZB5fZVD4gz4qW6TxyyNz33wVfW9hQ, address:145.79.227.27:4701): protocol: rpc: dial tcp 145.79.227.27:4701: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:40.460512619 (296h37m38.057283649s) [noise=true , address=145.79.227.40:4062 , piece_id=BXDV657N73AWJLF54QJWYQ5Q6KTO62AKXVYVRCYKPXYNGYDH6R4A, seg_pos={0 0}, upload=332226]: Done uploading piece. err: ecclient: upload failed (node:12N61Va6p4WU9cmUz65CkCDKr54LFtJhh8HJGYPVt8cJTRoWLhb, address:145.79.227.40:4062): protocol: rpc: dial tcp 145.79.227.40:4062: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:40.460673235 (296h37m38.057444267s) [noise=true , address=145.79.227.36:4540 , piece_id=NCP3622NO7K3ZHBNOZVWRRCRHIQKHW5RN7N5OQCLL2UPSODMA4ZQ, seg_pos={0 0}, upload=332226]: Done uploading piece. err: ecclient: upload failed (node:1P92CRZqnx1avdDo18tMY2X7FujkZugb4gU98AaErD13gX4cLA, address:145.79.227.36:4540): protocol: rpc: dial tcp 145.79.227.36:4540: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:40.460706522 (296h37m38.057477535s) [seg_pos={0 0}, upload=332226]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Mar 18 19:37:40.478800721 (296h37m38.075571752s) [upload=332226]: Done uploading segments... Mar 18 19:37:40.478897894 (296h37m38.075668927s) [upload=332226]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 1.294239613s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 1m0.023660394s trace: https://jaeger.dev.storj.io/trace/504da329c61bc56 FAIL (2026-03-18 19:37:40 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== in upload segment... Mar 18 19:36:40.519062491 (296h36m38.115833524s) [seg_pos={0 0}, upload=332226]: Done begin upload segment. Mar 18 19:36:40.519122972 (296h36m38.115893995s) [noise=true , address=167.17.72.70:4756 , piece_id=RF4MNRT5B5WSDQ5OE4YIDS6FJJRKMGWEEDMPZU4GUESEDQZUYS4A, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.519158535 (296h36m38.115929568s) [noise=true , address=193.149.164.70:4137 , piece_id=IK2NPZKZECM55AGMLYHMGRML4RT677DLAQK2RUPNLTS2DPWZESJA, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.519952691 (296h36m38.116723717s) [noise=true , address=145.79.227.39:4045 , piece_id=ONYBRZMCR6WZIAFOJ43XMTLVGBCH7GYTGBT5NBSY6U26WPOR72AQ, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.520469473 (296h36m38.117240506s) [noise=true , address=77.90.31.17:4783 , piece_id=K7YEUHBNATYTZ3AGF3PJB27L6SFI5J5PLDO3F6WTIHW45WBBOMGQ, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.520531096 (296h36m38.117302138s) [noise=true , address=77.90.31.13:4640 , piece_id=JWLWW3WC5WL3W7C774PU5EFQJJVRHGRDXTIJLIWQFLMBSV4BFSVQ, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.521025199 (296h36m38.117796226s) [noise=true , address=145.79.227.2:4135 , piece_id=QS2HTEINCAU5IRXJHWJPZPOUXUER7LV2LA7GLPRMLXZKZM5PRPCQ, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.521195168 (296h36m38.117966191s) [noise=true , address=145.79.227.27:4701 , piece_id=GIZEBKJF5M7QBMLYPN3WQUIJYVHHD6HXEVPGKAIIOLA273AWTPPQ, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.521517234 (296h36m38.118288257s) [noise=true , address=145.79.227.37:4589 , piece_id=Z3ZPG2RWAH375JWNP3Q42MJHOG4AVZBQOAS27ISLSB7TOVFUSNHQ, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.521721504 (296h36m38.118492529s) [noise=true , address=64.49.13.137:4503 , piece_id=CGB3TCGOS6Y3UHUJS7MLV24FUDN3JVHPXQ2QIIETRONZ4H7C6S3A, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.522279589 (296h36m38.119050616s) [noise=true , address=79.127.219.46:4911 , piece_id=RGSZ7VYJJCBNEPJZ6FJ7BM6BQGOSBZQLRWXHNNB4C2C3PSN43NNA, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.523062176 (296h36m38.119833208s) [noise=true , address=77.90.31.12:4595 , piece_id=ZTJWMLII2CK4LHI3TXZWZIADFIJQAD2E5ID44UEUQFRGMG3EU54A, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.523708767 (296h36m38.12047979s) [noise=true , address=145.79.227.36:4540 , piece_id=NCP3622NO7K3ZHBNOZVWRRCRHIQKHW5RN7N5OQCLL2UPSODMA4ZQ, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.524314790 (296h36m38.121085826s) [noise=true , address=145.79.227.40:4062 , piece_id=BXDV657N73AWJLF54QJWYQ5Q6KTO62AKXVYVRCYKPXYNGYDH6R4A, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.524789568 (296h36m38.121560591s) [noise=true , address=193.149.164.67:4008 , piece_id=Y6NJEIGE5JWOBWVMTQO4BYBSY2BX7OE45TKJI3BVX7KKQYMR3Z5Q, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.525324320 (296h36m38.122095333s) [noise=true , address=79.127.205.234:4206 , piece_id=233HW3UMHJLRTISXVCO2PR3UREECB63KELEBX4ZLYGZ7JPGRGQLA, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.525779446 (296h36m38.122550472s) [noise=true , address=79.127.219.45:4876 , piece_id=3TTZKRJ5BQF6IMQ6SRTH7DK2ZX7LXJLDPYDLIBIERU4WSRCYIWFQ, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.526254332 (296h36m38.123025355s) [noise=true , address=167.17.72.71:4784 , piece_id=ZBUUOXLEZQ353DQOY3ZSKI22H4T6VPHCDFMTQPMLGZBHYUYUXIBA, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.526725151 (296h36m38.123496178s) [noise=true , address=145.79.227.52:4475 , piece_id=QFVWGT7WIWXIHX3OXZKXWRZ3HNKF4EXN7C6U2OFSQMZF56QCCHQA, seg_pos={0 0}, upload=332226]: Uploading piece... Mar 18 19:36:40.589597769 (296h36m38.186368792s) [upload=332226]: Next returned nil segment Mar 18 19:36:40.589617873 (296h36m38.186388903s) [upload=332226]: Waiting for error group managing segments... Mar 18 19:36:40.594936043 (296h36m38.191707073s) [noise=true , address=77.90.31.12:4595 , piece_id=ZTJWMLII2CK4LHI3TXZWZIADFIJQAD2E5ID44UEUQFRGMG3EU54A, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:36:40.597084865 (296h36m38.193855887s) [noise=true , address=79.127.219.45:4876 , piece_id=3TTZKRJ5BQF6IMQ6SRTH7DK2ZX7LXJLDPYDLIBIERU4WSRCYIWFQ, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:36:40.610365162 (296h36m38.207136186s) [noise=true , address=77.90.31.13:4640 , piece_id=JWLWW3WC5WL3W7C774PU5EFQJJVRHGRDXTIJLIWQFLMBSV4BFSVQ, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:36:40.623540288 (296h36m38.220311316s) [noise=true , address=77.90.31.17:4783 , piece_id=K7YEUHBNATYTZ3AGF3PJB27L6SFI5J5PLDO3F6WTIHW45WBBOMGQ, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:36:40.637765186 (296h36m38.234536209s) [noise=true , address=79.127.219.46:4911 , piece_id=RGSZ7VYJJCBNEPJZ6FJ7BM6BQGOSBZQLRWXHNNB4C2C3PSN43NNA, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:36:40.849102006 (296h36m38.445873019s) [noise=true , address=167.17.72.70:4756 , piece_id=RF4MNRT5B5WSDQ5OE4YIDS6FJJRKMGWEEDMPZU4GUESEDQZUYS4A, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:36:40.854370150 (296h36m38.451141177s) [noise=true , address=167.17.72.71:4784 , piece_id=ZBUUOXLEZQ353DQOY3ZSKI22H4T6VPHCDFMTQPMLGZBHYUYUXIBA, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:36:41.113067742 (296h36m38.709838765s) [noise=true , address=193.149.164.67:4008 , piece_id=Y6NJEIGE5JWOBWVMTQO4BYBSY2BX7OE45TKJI3BVX7KKQYMR3Z5Q, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:36:41.117252653 (296h36m38.714023687s) [noise=true , address=79.127.205.234:4206 , piece_id=233HW3UMHJLRTISXVCO2PR3UREECB63KELEBX4ZLYGZ7JPGRGQLA, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:36:41.118717458 (296h36m38.715488481s) [noise=true , address=64.49.13.137:4503 , piece_id=CGB3TCGOS6Y3UHUJS7MLV24FUDN3JVHPXQ2QIIETRONZ4H7C6S3A, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:36:41.123965737 (296h36m38.720736764s) [noise=true , address=193.149.164.70:4137 , piece_id=IK2NPZKZECM55AGMLYHMGRML4RT677DLAQK2RUPNLTS2DPWZESJA, seg_pos={0 0}, upload=332226]: Done uploading piece. err: <nil> Mar 18 19:37:40.459260224 (296h37m38.056031257s) [noise=true , address=145.79.227.37:4589 , piece_id=Z3ZPG2RWAH375JWNP3Q42MJHOG4AVZBQOAS27ISLSB7TOVFUSNHQ, seg_pos={0 0}, upload=332226]: Done uploading piece. err: ecclient: upload failed (node:12ChutvH6jC7eWE7dDE9Cmsn8EDo6ecHum8dG9NBvEcdphiJgbi, address:145.79.227.37:4589): protocol: rpc: dial tcp 145.79.227.37:4589: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:40.459832101 (296h37m38.056603126s) [noise=true , address=145.79.227.52:4475 , piece_id=QFVWGT7WIWXIHX3OXZKXWRZ3HNKF4EXN7C6U2OFSQMZF56QCCHQA, seg_pos={0 0}, upload=332226]: Done uploading piece. err: ecclient: upload failed (node:12UYjE6PEB9gZKwTKcnx23ykF4jDMJzLQoKmiFiejWew5mirdCj, address:145.79.227.52:4475): protocol: rpc: dial tcp 145.79.227.52:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:40.460091051 (296h37m38.056862082s) [noise=true , address=145.79.227.2:4135 , piece_id=QS2HTEINCAU5IRXJHWJPZPOUXUER7LV2LA7GLPRMLXZKZM5PRPCQ, seg_pos={0 0}, upload=332226]: Done uploading piece. err: ecclient: upload failed (node:1uEBWRaMH4mLApUcgHyHQmGxxbyXnYUScg5Z2on9LHESYkqj3S, address:145.79.227.2:4135): protocol: rpc: dial tcp 145.79.227.2:4135: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:40.460212199 (296h37m38.056983231s) [noise=true , address=145.79.227.39:4045 , piece_id=ONYBRZMCR6WZIAFOJ43XMTLVGBCH7GYTGBT5NBSY6U26WPOR72AQ, seg_pos={0 0}, upload=332226]: Done uploading piece. err: ecclient: upload failed (node:1j6J2T4aX3bmjtP9J379mNQBFDp3vQZeAvmUFEYVfYZFknjrTp, address:145.79.227.39:4045): protocol: rpc: dial tcp 145.79.227.39:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:40.460371799 (296h37m38.05714283s) [noise=true , address=145.79.227.27:4701 , piece_id=GIZEBKJF5M7QBMLYPN3WQUIJYVHHD6HXEVPGKAIIOLA273AWTPPQ, seg_pos={0 0}, upload=332226]: Done uploading piece. err: ecclient: upload failed (node:1weCZWAtZRRYdeuPE8TjnZB5fZVD4gz4qW6TxyyNz33wVfW9hQ, address:145.79.227.27:4701): protocol: rpc: dial tcp 145.79.227.27:4701: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:40.460512619 (296h37m38.057283649s) [noise=true , address=145.79.227.40:4062 , piece_id=BXDV657N73AWJLF54QJWYQ5Q6KTO62AKXVYVRCYKPXYNGYDH6R4A, seg_pos={0 0}, upload=332226]: Done uploading piece. err: ecclient: upload failed (node:12N61Va6p4WU9cmUz65CkCDKr54LFtJhh8HJGYPVt8cJTRoWLhb, address:145.79.227.40:4062): protocol: rpc: dial tcp 145.79.227.40:4062: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:40.460673235 (296h37m38.057444267s) [noise=true , address=145.79.227.36:4540 , piece_id=NCP3622NO7K3ZHBNOZVWRRCRHIQKHW5RN7N5OQCLL2UPSODMA4ZQ, seg_pos={0 0}, upload=332226]: Done uploading piece. err: ecclient: upload failed (node:1P92CRZqnx1avdDo18tMY2X7FujkZugb4gU98AaErD13gX4cLA, address:145.79.227.36:4540): protocol: rpc: dial tcp 145.79.227.36:4540: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:40.460706522 (296h37m38.057477535s) [seg_pos={0 0}, upload=332226]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Mar 18 19:37:40.478800721 (296h37m38.075571752s) [upload=332226]: Done uploading segments... Mar 18 19:37:40.478897894 (296h37m38.075668927s) [upload=332226]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 1m0.023679363s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 1.048412361s trace: https://jaeger.dev.storj.io/trace/15ce03727facea57 FAIL (2026-03-18 19:37:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Mar 18 19:36:10.500432381 (296h36m8.097203414s) [upload=332219]: Starting upload Mar 18 19:36:10.500543799 (296h36m8.097314825s) [upload=332219]: Done starting upload Mar 18 19:36:10.500649790 (296h36m8.097420808s) [upload=332219]: Uploading segments... Mar 18 19:36:10.500874959 (296h36m8.097645973s) [upload=332219]: Got next segment. Inline: false Mar 18 19:36:10.500932640 (296h36m8.097703652s) [upload=332219]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 19:36:10.500938749 (296h36m8.097709772s) [upload=332219]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 19:36:10.530892350 (296h36m8.127663383s) [seg_pos={0 0}, upload=332219]: Begin upload segment... Mar 18 19:36:10.530984837 (296h36m8.127755871s) [seg_pos={0 0}, upload=332219]: Done begin upload segment. Mar 18 19:36:10.531072762 (296h36m8.127843796s) [noise=true , address=145.79.227.52:4458 , piece_id=SH4VV6STCFPDZ4QQUKDYJKQIH6Z7X4HPFEGG4UHLAMTX7U54K53Q, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.531740352 (296h36m8.128511384s) [noise=true , address=79.127.205.235:4272 , piece_id=4LY4OR3XG7TPI63P2Q5KVLX5LMU3XKBH4C3SMD3SNZRW3EGGJHAA, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.531870513 (296h36m8.128641539s) [noise=true , address=79.127.219.38:4851 , piece_id=HAAHZMKR5BH5JBKIM2C73226FZRBVEZESYDQATE6ZJNKHH4ZPCDQ, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.532746824 (296h36m8.129517851s) [noise=true , address=77.90.31.21:4903 , piece_id=JBH363RHO4FJ2G4HJIDRQM4U3H6BW2ON7DLYZLDL6WUEWKGCNZCQ, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.532993109 (296h36m8.129764133s) [noise=true , address=77.90.31.17:4798 , piece_id=PRUFHXZXFJSV456IJM4YGC6FO65S4W24K5QGXWSAJZC4H77MWYBQ, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.533827706 (296h36m8.130598738s) [noise=true , address=145.79.227.3:4156 , piece_id=E67ZTTSMDV2WBUAI4O7RUQ2QWHMA7VSGRSF32NKD4AZRP6FXKLVA, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.533923098 (296h36m8.130694124s) [noise=true , address=79.127.219.46:4919 , piece_id=CWWJPNSKGWZRN5JUBINAAV2TFLDTBH7DZRMV6V7SV7ZBAWROYNKQ, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.534804584 (296h36m8.131575613s) [noise=true , address=145.79.227.36:4540 , piece_id=C6JUTP5TWCMIIHQQOPCXNDTNNCXN5Y37IZQAIK6O6RHELC3ZO6AA, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.534867539 (296h36m8.131638564s) [noise=true , address=193.149.164.70:4106 , piece_id=RBCC4FVQZBNI74A4SCXY2OFDBBNMYO5FH32CPHPZDN3XOG2KEJLQ, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.535728720 (296h36m8.132499754s) [noise=true , address=145.79.227.22:4538 , piece_id=Y2MRTJ6FUF7IY6TA3RQKMTHHDJ6E62JUZ5I7K7J5D5BRR2HPPH7A, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.535783456 (296h36m8.132554487s) [noise=true , address=193.149.164.69:4081 , piece_id=JORHOCGDU6SNE57F4WVH4AUVJ3HAX3V6MSB7L4YNBWKLXRC7K5LQ, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.536539101 (296h36m8.133310129s) [noise=true , address=193.149.164.68:4060 , piece_id=XY5RJPNM5BM27A6MJDK26WSTBHC3RJBZSRXS3VS32YUT2M7Q6UPA, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.536922919 (296h36m8.133693949s) [noise=true , address=145.79.227.25:4635 , piece_id=T7LMGZK2TPA5GXWJZG3OYECOPFIY2EXLYIISGOFMVX7HLUCYHPTQ, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.537283766 (296h36m8.134054813s) [noise=true , address=79.127.219.45:4868 , piece_id=LU76MTXKI7UVDV5VHXLFGZO3CLWKEQJGR463E3J2LDLDB57AA3ZQ, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.538845517 (296h36m8.135616541s) [noise=true , address=77.90.31.16:4752 , piece_id=MH7M34YWITMHOLYDHZAA5HZOVME6A6NSZDUALQJRPQNAMP22PMAQ, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.539332130 (296h36m8.136103158s) [noise=true , address=167.17.72.69:4719 , piece_id=2AY2C35O5IKGH45UWFYDHSA6QKSCQVVIK3WCNKTNOWCCKGWQY5WA, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.539744551 (296h36m8.136515577s) [noise=true , address=167.17.72.68:4661 , piece_id=VWCENPL2O42UCC6JOUFH337C2DDY3CYNTPCKPNGFPRWLIYE7YUCA, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.540083638 (296h36m8.136854667s) [noise=true , address=45.91.116.14:4209 , piece_id=NUOPZWUJ7QTBQ3RWNRYC3H4I2DACNCYZ75XQYOBLUSITNU2PK3LQ, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.601600388 (296h36m8.198371421s) [upload=332219]: Next returned nil segment Mar 18 19:36:10.601617944 (296h36m8.198388973s) [upload=332219]: Waiting for error group managing segments... Mar 18 19:36:10.606706422 (296h36m8.20347745s) [noise=true , address=77.90.31.16:4752 , piece_id=MH7M34YWITMHOLYDHZAA5HZOVME6A6NSZDUALQJRPQNAMP22PMAQ, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:10.607550275 (296h36m8.204321303s) [noise=true , address=79.127.219.45:4868 , piece_id=LU76MTXKI7UVDV5VHXLFGZO3CLWKEQJGR463E3J2LDLDB57AA3ZQ, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:10.608806703 (296h36m8.205577728s) [noise=true , address=79.127.219.38:4851 , piece_id=HAAHZMKR5BH5JBKIM2C73226FZRBVEZESYDQATE6ZJNKHH4ZPCDQ, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:10.622912404 (296h36m8.219683443s) [noise=true , address=45.91.116.14:4209 , piece_id=NUOPZWUJ7QTBQ3RWNRYC3H4I2DACNCYZ75XQYOBLUSITNU2PK3LQ, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:10.623232794 (296h36m8.220003819s) [noise=true , address=77.90.31.17:4798 , piece_id=PRUFHXZXFJSV456IJM4YGC6FO65S4W24K5QGXWSAJZC4H77MWYBQ, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:10.623919545 (296h36m8.220690574s) [noise=true , address=79.127.219.46:4919 , piece_id=CWWJPNSKGWZRN5JUBINAAV2TFLDTBH7DZRMV6V7SV7ZBAWROYNKQ, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:10.669238480 (296h36m8.266009504s) [noise=true , address=77.90.31.21:4903 , piece_id=JBH363RHO4FJ2G4HJIDRQM4U3H6BW2ON7DLYZLDL6WUEWKGCNZCQ, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:10.849192400 (296h36m8.44596344s) [noise=true , address=167.17.72.68:4661 , piece_id=VWCENPL2O42UCC6JOUFH337C2DDY3CYNTPCKPNGFPRWLIYE7YUCA, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:10.899652863 (296h36m8.496423891s) [noise=true , address=167.17.72.69:4719 , piece_id=2AY2C35O5IKGH45UWFYDHSA6QKSCQVVIK3WCNKTNOWCCKGWQY5WA, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:11.118324074 (296h36m8.715095097s) [noise=true , address=79.127.205.235:4272 , piece_id=4LY4OR3XG7TPI63P2Q5KVLX5LMU3XKBH4C3SMD3SNZRW3EGGJHAA, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:11.127553350 (296h36m8.724324364s) [noise=true , address=193.149.164.69:4081 , piece_id=JORHOCGDU6SNE57F4WVH4AUVJ3HAX3V6MSB7L4YNBWKLXRC7K5LQ, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:11.145684011 (296h36m8.742455034s) [noise=true , address=193.149.164.68:4060 , piece_id=XY5RJPNM5BM27A6MJDK26WSTBHC3RJBZSRXS3VS32YUT2M7Q6UPA, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:11.146892159 (296h36m8.743663183s) [noise=true , address=193.149.164.70:4106 , piece_id=RBCC4FVQZBNI74A4SCXY2OFDBBNMYO5FH32CPHPZDN3XOG2KEJLQ, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:37:10.457013911 (296h37m8.05378494s) [noise=true , address=145.79.227.52:4458 , piece_id=SH4VV6STCFPDZ4QQUKDYJKQIH6Z7X4HPFEGG4UHLAMTX7U54K53Q, seg_pos={0 0}, upload=332219]: Done uploading piece. err: ecclient: upload failed (node:1TMPdLfCbmgi3GqDDg6AUvTCFPzbCCgwTAn9JJBrYJRnbkKtpb, address:145.79.227.52:4458): protocol: rpc: dial tcp 145.79.227.52:4458: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:10.457080518 (296h37m8.053851548s) [noise=true , address=145.79.227.3:4156 , piece_id=E67ZTTSMDV2WBUAI4O7RUQ2QWHMA7VSGRSF32NKD4AZRP6FXKLVA, seg_pos={0 0}, upload=332219]: Done uploading piece. err: ecclient: upload failed (node:12JTrtKCNHFUY91Ktaa7v7eAWe4GHssKSeofrg3diu8isHJnmy4, address:145.79.227.3:4156): protocol: rpc: dial tcp 145.79.227.3:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:10.457355833 (296h37m8.054126865s) [noise=true , address=145.79.227.36:4540 , piece_id=C6JUTP5TWCMIIHQQOPCXNDTNNCXN5Y37IZQAIK6O6RHELC3ZO6AA, seg_pos={0 0}, upload=332219]: Done uploading piece. err: ecclient: upload failed (node:1P92CRZqnx1avdDo18tMY2X7FujkZugb4gU98AaErD13gX4cLA, address:145.79.227.36:4540): protocol: rpc: dial tcp 145.79.227.36:4540: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:10.458754175 (296h37m8.055525209s) [noise=true , address=145.79.227.22:4538 , piece_id=Y2MRTJ6FUF7IY6TA3RQKMTHHDJ6E62JUZ5I7K7J5D5BRR2HPPH7A, seg_pos={0 0}, upload=332219]: Done uploading piece. err: ecclient: upload failed (node:1kNFzPpkDtwy4VFyzeFQReorJPEVsmdDAd8ULT8ZKoGde57P3v, address:145.79.227.22:4538): protocol: rpc: dial tcp 145.79.227.22:4538: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:10.459003879 (296h37m8.055774913s) [noise=true , address=145.79.227.25:4635 , piece_id=T7LMGZK2TPA5GXWJZG3OYECOPFIY2EXLYIISGOFMVX7HLUCYHPTQ, seg_pos={0 0}, upload=332219]: Done uploading piece. err: ecclient: upload failed (node:12jyQ3G4H8fYWdP6RiREP93PzZq3G9wm2BKKk2tZSfcmyopP1a2, address:145.79.227.25:4635): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:37:10.459074189 (296h37m8.055845199s) [seg_pos={0 0}, upload=332219]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Mar 18 19:37:10.482165416 (296h37m8.078936449s) [upload=332219]: Done uploading segments... Mar 18 19:37:10.482224140 (296h37m8.078995174s) [upload=332219]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.048438076s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.02898262s trace: https://jaeger.dev.storj.io/trace/15ce03727facea57 FAIL (2026-03-18 19:37:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Mar 18 19:36:10.500432381 (296h36m8.097203414s) [upload=332219]: Starting upload Mar 18 19:36:10.500543799 (296h36m8.097314825s) [upload=332219]: Done starting upload Mar 18 19:36:10.500649790 (296h36m8.097420808s) [upload=332219]: Uploading segments... Mar 18 19:36:10.500874959 (296h36m8.097645973s) [upload=332219]: Got next segment. Inline: false Mar 18 19:36:10.500932640 (296h36m8.097703652s) [upload=332219]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 19:36:10.500938749 (296h36m8.097709772s) [upload=332219]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 19:36:10.530892350 (296h36m8.127663383s) [seg_pos={0 0}, upload=332219]: Begin upload segment... Mar 18 19:36:10.530984837 (296h36m8.127755871s) [seg_pos={0 0}, upload=332219]: Done begin upload segment. Mar 18 19:36:10.531072762 (296h36m8.127843796s) [noise=true , address=145.79.227.52:4458 , piece_id=SH4VV6STCFPDZ4QQUKDYJKQIH6Z7X4HPFEGG4UHLAMTX7U54K53Q, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.531740352 (296h36m8.128511384s) [noise=true , address=79.127.205.235:4272 , piece_id=4LY4OR3XG7TPI63P2Q5KVLX5LMU3XKBH4C3SMD3SNZRW3EGGJHAA, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.531870513 (296h36m8.128641539s) [noise=true , address=79.127.219.38:4851 , piece_id=HAAHZMKR5BH5JBKIM2C73226FZRBVEZESYDQATE6ZJNKHH4ZPCDQ, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.532746824 (296h36m8.129517851s) [noise=true , address=77.90.31.21:4903 , piece_id=JBH363RHO4FJ2G4HJIDRQM4U3H6BW2ON7DLYZLDL6WUEWKGCNZCQ, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.532993109 (296h36m8.129764133s) [noise=true , address=77.90.31.17:4798 , piece_id=PRUFHXZXFJSV456IJM4YGC6FO65S4W24K5QGXWSAJZC4H77MWYBQ, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.533827706 (296h36m8.130598738s) [noise=true , address=145.79.227.3:4156 , piece_id=E67ZTTSMDV2WBUAI4O7RUQ2QWHMA7VSGRSF32NKD4AZRP6FXKLVA, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.533923098 (296h36m8.130694124s) [noise=true , address=79.127.219.46:4919 , piece_id=CWWJPNSKGWZRN5JUBINAAV2TFLDTBH7DZRMV6V7SV7ZBAWROYNKQ, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.534804584 (296h36m8.131575613s) [noise=true , address=145.79.227.36:4540 , piece_id=C6JUTP5TWCMIIHQQOPCXNDTNNCXN5Y37IZQAIK6O6RHELC3ZO6AA, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.534867539 (296h36m8.131638564s) [noise=true , address=193.149.164.70:4106 , piece_id=RBCC4FVQZBNI74A4SCXY2OFDBBNMYO5FH32CPHPZDN3XOG2KEJLQ, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.535728720 (296h36m8.132499754s) [noise=true , address=145.79.227.22:4538 , piece_id=Y2MRTJ6FUF7IY6TA3RQKMTHHDJ6E62JUZ5I7K7J5D5BRR2HPPH7A, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.535783456 (296h36m8.132554487s) [noise=true , address=193.149.164.69:4081 , piece_id=JORHOCGDU6SNE57F4WVH4AUVJ3HAX3V6MSB7L4YNBWKLXRC7K5LQ, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.536539101 (296h36m8.133310129s) [noise=true , address=193.149.164.68:4060 , piece_id=XY5RJPNM5BM27A6MJDK26WSTBHC3RJBZSRXS3VS32YUT2M7Q6UPA, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.536922919 (296h36m8.133693949s) [noise=true , address=145.79.227.25:4635 , piece_id=T7LMGZK2TPA5GXWJZG3OYECOPFIY2EXLYIISGOFMVX7HLUCYHPTQ, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.537283766 (296h36m8.134054813s) [noise=true , address=79.127.219.45:4868 , piece_id=LU76MTXKI7UVDV5VHXLFGZO3CLWKEQJGR463E3J2LDLDB57AA3ZQ, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.538845517 (296h36m8.135616541s) [noise=true , address=77.90.31.16:4752 , piece_id=MH7M34YWITMHOLYDHZAA5HZOVME6A6NSZDUALQJRPQNAMP22PMAQ, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.539332130 (296h36m8.136103158s) [noise=true , address=167.17.72.69:4719 , piece_id=2AY2C35O5IKGH45UWFYDHSA6QKSCQVVIK3WCNKTNOWCCKGWQY5WA, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.539744551 (296h36m8.136515577s) [noise=true , address=167.17.72.68:4661 , piece_id=VWCENPL2O42UCC6JOUFH337C2DDY3CYNTPCKPNGFPRWLIYE7YUCA, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.540083638 (296h36m8.136854667s) [noise=true , address=45.91.116.14:4209 , piece_id=NUOPZWUJ7QTBQ3RWNRYC3H4I2DACNCYZ75XQYOBLUSITNU2PK3LQ, seg_pos={0 0}, upload=332219]: Uploading piece... Mar 18 19:36:10.601600388 (296h36m8.198371421s) [upload=332219]: Next returned nil segment Mar 18 19:36:10.601617944 (296h36m8.198388973s) [upload=332219]: Waiting for error group managing segments... Mar 18 19:36:10.606706422 (296h36m8.20347745s) [noise=true , address=77.90.31.16:4752 , piece_id=MH7M34YWITMHOLYDHZAA5HZOVME6A6NSZDUALQJRPQNAMP22PMAQ, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:10.607550275 (296h36m8.204321303s) [noise=true , address=79.127.219.45:4868 , piece_id=LU76MTXKI7UVDV5VHXLFGZO3CLWKEQJGR463E3J2LDLDB57AA3ZQ, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:10.608806703 (296h36m8.205577728s) [noise=true , address=79.127.219.38:4851 , piece_id=HAAHZMKR5BH5JBKIM2C73226FZRBVEZESYDQATE6ZJNKHH4ZPCDQ, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:10.622912404 (296h36m8.219683443s) [noise=true , address=45.91.116.14:4209 , piece_id=NUOPZWUJ7QTBQ3RWNRYC3H4I2DACNCYZ75XQYOBLUSITNU2PK3LQ, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:10.623232794 (296h36m8.220003819s) [noise=true , address=77.90.31.17:4798 , piece_id=PRUFHXZXFJSV456IJM4YGC6FO65S4W24K5QGXWSAJZC4H77MWYBQ, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:10.623919545 (296h36m8.220690574s) [noise=true , address=79.127.219.46:4919 , piece_id=CWWJPNSKGWZRN5JUBINAAV2TFLDTBH7DZRMV6V7SV7ZBAWROYNKQ, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:10.669238480 (296h36m8.266009504s) [noise=true , address=77.90.31.21:4903 , piece_id=JBH363RHO4FJ2G4HJIDRQM4U3H6BW2ON7DLYZLDL6WUEWKGCNZCQ, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:10.849192400 (296h36m8.44596344s) [noise=true , address=167.17.72.68:4661 , piece_id=VWCENPL2O42UCC6JOUFH337C2DDY3CYNTPCKPNGFPRWLIYE7YUCA, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:10.899652863 (296h36m8.496423891s) [noise=true , address=167.17.72.69:4719 , piece_id=2AY2C35O5IKGH45UWFYDHSA6QKSCQVVIK3WCNKTNOWCCKGWQY5WA, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:11.118324074 (296h36m8.715095097s) [noise=true , address=79.127.205.235:4272 , piece_id=4LY4OR3XG7TPI63P2Q5KVLX5LMU3XKBH4C3SMD3SNZRW3EGGJHAA, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:11.127553350 (296h36m8.724324364s) [noise=true , address=193.149.164.69:4081 , piece_id=JORHOCGDU6SNE57F4WVH4AUVJ3HAX3V6MSB7L4YNBWKLXRC7K5LQ, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:11.145684011 (296h36m8.742455034s) [noise=true , address=193.149.164.68:4060 , piece_id=XY5RJPNM5BM27A6MJDK26WSTBHC3RJBZSRXS3VS32YUT2M7Q6UPA, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:36:11.146892159 (296h36m8.743663183s) [noise=true , address=193.149.164.70:4106 , piece_id=RBCC4FVQZBNI74A4SCXY2OFDBBNMYO5FH32CPHPZDN3XOG2KEJLQ, seg_pos={0 0}, upload=332219]: Done uploading piece. err: <nil> Mar 18 19:37:10.457013911 (296h37m8.05378494s) [noise=true , address=145.79.227.52:4458 , piece_id=SH4VV6STCFPDZ4QQUKDYJKQIH6Z7X4HPFEGG4UHLAMTX7U54K53Q, seg_pos={0 0}, upload=332219]: Done uploading piece. err: ecclient: upload failed (node:1TMPdLfCbmgi3GqDDg6AUvTCFPzbCCgwTAn9JJBrYJRnbkKtpb, address:145.79.227.52:4458): protocol: rpc: dial tcp 145.79.227.52:4458: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:10.457080518 (296h37m8.053851548s) [noise=true , address=145.79.227.3:4156 , piece_id=E67ZTTSMDV2WBUAI4O7RUQ2QWHMA7VSGRSF32NKD4AZRP6FXKLVA, seg_pos={0 0}, upload=332219]: Done uploading piece. err: ecclient: upload failed (node:12JTrtKCNHFUY91Ktaa7v7eAWe4GHssKSeofrg3diu8isHJnmy4, address:145.79.227.3:4156): protocol: rpc: dial tcp 145.79.227.3:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:10.457355833 (296h37m8.054126865s) [noise=true , address=145.79.227.36:4540 , piece_id=C6JUTP5TWCMIIHQQOPCXNDTNNCXN5Y37IZQAIK6O6RHELC3ZO6AA, seg_pos={0 0}, upload=332219]: Done uploading piece. err: ecclient: upload failed (node:1P92CRZqnx1avdDo18tMY2X7FujkZugb4gU98AaErD13gX4cLA, address:145.79.227.36:4540): protocol: rpc: dial tcp 145.79.227.36:4540: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:10.458754175 (296h37m8.055525209s) [noise=true , address=145.79.227.22:4538 , piece_id=Y2MRTJ6FUF7IY6TA3RQKMTHHDJ6E62JUZ5I7K7J5D5BRR2HPPH7A, seg_pos={0 0}, upload=332219]: Done uploading piece. err: ecclient: upload failed (node:1kNFzPpkDtwy4VFyzeFQReorJPEVsmdDAd8ULT8ZKoGde57P3v, address:145.79.227.22:4538): protocol: rpc: dial tcp 145.79.227.22:4538: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:10.459003879 (296h37m8.055774913s) [noise=true , address=145.79.227.25:4635 , piece_id=T7LMGZK2TPA5GXWJZG3OYECOPFIY2EXLYIISGOFMVX7HLUCYHPTQ, seg_pos={0 0}, upload=332219]: Done uploading piece. err: ecclient: upload failed (node:12jyQ3G4H8fYWdP6RiREP93PzZq3G9wm2BKKk2tZSfcmyopP1a2, address:145.79.227.25:4635): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:37:10.459074189 (296h37m8.055845199s) [seg_pos={0 0}, upload=332219]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Mar 18 19:37:10.482165416 (296h37m8.078936449s) [upload=332219]: Done uploading segments... Mar 18 19:37:10.482224140 (296h37m8.078995174s) [upload=332219]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.029022884s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 7 out of 60 is failed (threshold=6) [.....................................................XXXXXXX] duration: 5.00149357s trace: https://jaeger.dev.storj.io/trace/69466d136cbd49bb FAIL (2026-03-18 19:37:09 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001519181s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.......................................................XXXXX] duration: 30.03239622s trace: https://jaeger.dev.storj.io/trace/1f184eae0dfc09e1 FAIL (2026-03-18 19:37:06 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== ar 18 19:36:36.555798843 (296h36m34.152569869s) [seg_pos={0 0}, upload=332225]: Begin upload segment... Mar 18 19:36:36.555886626 (296h36m34.152657652s) [seg_pos={0 0}, upload=332225]: Done begin upload segment. Mar 18 19:36:36.555918660 (296h36m34.152689698s) [noise=true , address=145.79.227.49:4339 , piece_id=JVN56YGAYQP6IZHGKKWH7MP34W67O5LQVJUWT7I6YQOEYSLM2SEQ, seg_pos={0 0}, upload=332225]: Uploading piece... Mar 18 19:36:36.555950089 (296h36m34.152721115s) [noise=true , address=145.79.227.3:4154 , piece_id=3F5EJHAWQ36TBKYDLIRRAKMSE5H7QY3VC5NNGHTAZZKRLMP3QJ3Q, seg_pos={0 0}, upload=332225]: Uploading piece... Mar 18 19:36:36.556538069 (296h36m34.153309109s) [noise=true , address=145.79.227.16:4911 , piece_id=DQWK6KFKJIZW2XEQP2ODMWC7GXUZ4AU2JW4SOOEVABLX2YCFD5CA, seg_pos={0 0}, upload=332225]: Uploading piece... Mar 18 19:36:36.556662241 (296h36m34.153433264s) [noise=true , address=145.79.227.10:4406 , piece_id=QMZIITNFGFX4VQLIIJ6AK6JGLBNPKEGKXIYLVDU7OZSQKYBWXXVA, seg_pos={0 0}, upload=332225]: Uploading piece... Mar 18 19:36:36.557091631 (296h36m34.153862663s) [noise=true , address=145.79.227.41:4090 , piece_id=CTVINOEMOGFNUYJRASO6ZDABGAXJFMWKSWEZ7AFDZWX3XLBN2LKQ, seg_pos={0 0}, upload=332225]: Uploading piece... Mar 18 19:36:36.557552462 (296h36m34.154323496s) [noise=true , address=45.91.116.6:4974 , piece_id=T6FN5AWZDZ27C2CHGWXLVK7C56GFICHB43WSUMSVKXQQ2EIBDXQA, seg_pos={0 0}, upload=332225]: Uploading piece... Mar 18 19:36:36.557937090 (296h36m34.154708122s) [noise=true , address=145.79.227.20:4441 , piece_id=W5AJTUTI47BPK7HHFXXDYA32LAZX7P4UMNTIDHMMH5DOUDAASL2Q, seg_pos={0 0}, upload=332225]: Uploading piece... Mar 18 19:36:36.558501594 (296h36m34.155272732s) [noise=true , address=145.79.227.51:4410 , piece_id=B5R5UAC3KSL4RUPTB7NHKHY2PUKMYDXY72FHXWSDECZND6JGVD7A, seg_pos={0 0}, upload=332225]: Uploading piece... Mar 18 19:36:36.558892733 (296h36m34.155663761s) [noise=true , address=145.79.227.23:4550 , piece_id=6PS777IFO4RO6J3S2VY3SPFKKI35J6FO4THHWERQXSGE5HRJPQIA, seg_pos={0 0}, upload=332225]: Uploading piece... Mar 18 19:36:36.559147119 (296h36m34.155918148s) [noise=true , address=45.91.116.12:4244 , piece_id=UFG6UM72HMH5MNLY6VLOTJBZPAMGP35M2HTYOZ76WNLEDCGAKBMQ, seg_pos={0 0}, upload=332225]: Uploading piece... Mar 18 19:36:36.559724631 (296h36m34.156495657s) [noise=true , address=45.91.116.4:4947 , piece_id=J5MOFM7A6ICM635YSGYKVBHVVJEZ6232GOCGM3XVCC6AATBQKX7A, seg_pos={0 0}, upload=332225]: Uploading piece... Mar 18 19:36:36.560229874 (296h36m34.157000897s) [noise=true , address=145.79.227.44:4196 , piece_id=3W4KZYPFSSKUQJS7ZEP4LKNZSOCCZUAG26V4MJB7H2HREUYYNXCQ, seg_pos={0 0}, upload=332225]: Uploading piece... Mar 18 19:36:36.560544898 (296h36m34.157315929s) [noise=true , address=145.79.227.6:4262 , piece_id=2XBKWGMENS5MXQHKLMH4Y3JJYJGEQRXE7BS7MQTAHSV66J5OUG7Q, seg_pos={0 0}, upload=332225]: Uploading piece... Mar 18 19:36:36.561013038 (296h36m34.157784068s) [noise=true , address=77.90.31.19:4846 , piece_id=MF336XYQ6H45HZQHNW2Z62EINPEZCT4B3Z6PAFUNGDJPJKGV442A, seg_pos={0 0}, upload=332225]: Uploading piece... Mar 18 19:36:36.561386763 (296h36m34.15815779s) [noise=true , address=45.91.116.14:4224 , piece_id=ZTU44B6QKY2FGCKJSN3XXOT2ODVYAL4NMAIVSP3463EHTN7AJKBA, seg_pos={0 0}, upload=332225]: Uploading piece... Mar 18 19:36:36.561903005 (296h36m34.158674032s) [noise=true , address=145.79.227.26:4658 , piece_id=2TKHADME737JNXSGBDA7FCI5MPIUSIA5HV6M574PZTYPBJQ3M2BA, seg_pos={0 0}, upload=332225]: Uploading piece... Mar 18 19:36:36.626591323 (296h36m34.223362346s) [upload=332225]: Next returned nil segment Mar 18 19:36:36.626605410 (296h36m34.223376433s) [upload=332225]: Waiting for error group managing segments... Mar 18 19:36:36.632278187 (296h36m34.229049199s) [noise=true , address=45.91.116.12:4244 , piece_id=UFG6UM72HMH5MNLY6VLOTJBZPAMGP35M2HTYOZ76WNLEDCGAKBMQ, seg_pos={0 0}, upload=332225]: Done uploading piece. err: <nil> Mar 18 19:36:36.632509007 (296h36m34.229280031s) [noise=true , address=77.90.31.19:4846 , piece_id=MF336XYQ6H45HZQHNW2Z62EINPEZCT4B3Z6PAFUNGDJPJKGV442A, seg_pos={0 0}, upload=332225]: Done uploading piece. err: <nil> Mar 18 19:36:36.632834142 (296h36m34.229605155s) [noise=true , address=45.91.116.14:4224 , piece_id=ZTU44B6QKY2FGCKJSN3XXOT2ODVYAL4NMAIVSP3463EHTN7AJKBA, seg_pos={0 0}, upload=332225]: Done uploading piece. err: <nil> Mar 18 19:36:36.641083208 (296h36m34.237854236s) [noise=true , address=45.91.116.6:4974 , piece_id=T6FN5AWZDZ27C2CHGWXLVK7C56GFICHB43WSUMSVKXQQ2EIBDXQA, seg_pos={0 0}, upload=332225]: Done uploading piece. err: <nil> Mar 18 19:36:36.650441589 (296h36m34.247212602s) [noise=true , address=45.91.116.4:4947 , piece_id=J5MOFM7A6ICM635YSGYKVBHVVJEZ6232GOCGM3XVCC6AATBQKX7A, seg_pos={0 0}, upload=332225]: Done uploading piece. err: <nil> Mar 18 19:37:06.488720767 (296h37m4.085491798s) [noise=true , address=145.79.227.44:4196 , piece_id=3W4KZYPFSSKUQJS7ZEP4LKNZSOCCZUAG26V4MJB7H2HREUYYNXCQ, seg_pos={0 0}, upload=332225]: Done uploading piece. err: ecclient: upload failed (node:12Vs4grpszw7MhLJSMgCn7ikaL4jgwSB6XULfVgAt1Msy8hMcaL, address:145.79.227.44:4196): protocol: rpc: dial tcp 145.79.227.44:4196: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:06.489286886 (296h37m4.086057918s) [noise=true , address=145.79.227.49:4339 , piece_id=JVN56YGAYQP6IZHGKKWH7MP34W67O5LQVJUWT7I6YQOEYSLM2SEQ, seg_pos={0 0}, upload=332225]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:06.489637709 (296h37m4.086408736s) [noise=true , address=145.79.227.10:4406 , piece_id=QMZIITNFGFX4VQLIIJ6AK6JGLBNPKEGKXIYLVDU7OZSQKYBWXXVA, seg_pos={0 0}, upload=332225]: Done uploading piece. err: ecclient: upload failed (node:1KrucHjbKjhBZEUGd2p8cmfyCwm6rx4BVwwVhkpgnEv9EQiJRB, address:145.79.227.10:4406): protocol: rpc: dial tcp 145.79.227.10:4406: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:06.490605949 (296h37m4.087376982s) [noise=true , address=145.79.227.23:4550 , piece_id=6PS777IFO4RO6J3S2VY3SPFKKI35J6FO4THHWERQXSGE5HRJPQIA, seg_pos={0 0}, upload=332225]: Done uploading piece. err: ecclient: upload failed (node:12kn3uD4MCE56bN4MYYsGjDJd2qebgZ6VB6mavL711M8hoU8fbE, address:145.79.227.23:4550): protocol: rpc: dial tcp 145.79.227.23:4550: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:06.490776067 (296h37m4.087547099s) [noise=true , address=145.79.227.20:4441 , piece_id=W5AJTUTI47BPK7HHFXXDYA32LAZX7P4UMNTIDHMMH5DOUDAASL2Q, seg_pos={0 0}, upload=332225]: Done uploading piece. err: ecclient: upload failed (node:12LjGecRNJE1kq67iMtxZFhWeaUZpUogsGxbz4VXwnHdQtTnbBr, address:145.79.227.20:4441): protocol: rpc: dial tcp 145.79.227.20:4441: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:06.491557190 (296h37m4.088328225s) [noise=true , address=145.79.227.3:4154 , piece_id=3F5EJHAWQ36TBKYDLIRRAKMSE5H7QY3VC5NNGHTAZZKRLMP3QJ3Q, seg_pos={0 0}, upload=332225]: Done uploading piece. err: ecclient: upload failed (node:12S7Xsfnti3QuD9apBq1sgSjmahnAYA6LYbXqyckbLY4UU84ZQF, address:145.79.227.3:4154): protocol: rpc: dial tcp 145.79.227.3:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:06.491851037 (296h37m4.088622071s) [noise=true , address=145.79.227.16:4911 , piece_id=DQWK6KFKJIZW2XEQP2ODMWC7GXUZ4AU2JW4SOOEVABLX2YCFD5CA, seg_pos={0 0}, upload=332225]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: rpc: dial tcp 145.79.227.16:4911: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:06.493702212 (296h37m4.090473245s) [noise=true , address=145.79.227.26:4658 , piece_id=2TKHADME737JNXSGBDA7FCI5MPIUSIA5HV6M574PZTYPBJQ3M2BA, seg_pos={0 0}, upload=332225]: Done uploading piece. err: ecclient: upload failed (node:1gcSHRechDzTDwivRoAwNFfNj5Zjqvi6oJwVqzbCQinwgWtpPr, address:145.79.227.26:4658): protocol: rpc: dial tcp 145.79.227.26:4658: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:06.493841369 (296h37m4.0906124s) [noise=true , address=145.79.227.51:4410 , piece_id=B5R5UAC3KSL4RUPTB7NHKHY2PUKMYDXY72FHXWSDECZND6JGVD7A, seg_pos={0 0}, upload=332225]: Done uploading piece. err: ecclient: upload failed (node:12HWX3mC14qJg7WdCetaBtS2GcGQFhzSaqUbKrYFBHZKdyFAAgX, address:145.79.227.51:4410): protocol: rpc: dial tcp 145.79.227.51:4410: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:37:06.495990746 (296h37m4.092761779s) [noise=true , address=145.79.227.6:4262 , piece_id=2XBKWGMENS5MXQHKLMH4Y3JJYJGEQRXE7BS7MQTAHSV66J5OUG7Q, seg_pos={0 0}, upload=332225]: Done uploading piece. err: ecclient: upload failed (node:12nPw17QDTyrgDBBNKuunFWuDqPV5Rd2GpM7Loc6Ws9vAAZz22s, address:145.79.227.6:4262): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:37:06.497993586 (296h37m4.094764617s) [noise=true , address=145.79.227.41:4090 , piece_id=CTVINOEMOGFNUYJRASO6ZDABGAXJFMWKSWEZ7AFDZWX3XLBN2LKQ, seg_pos={0 0}, upload=332225]: Done uploading piece. err: ecclient: upload failed (node:12UUnoD9p5FC7rq9fMSzu1phkFpsnPsu4UYxC59MhpMjJ6wUC6F, address:145.79.227.41:4090): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:37:06.498072300 (296h37m4.094843323s) [seg_pos={0 0}, upload=332225]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Mar 18 19:37:06.518784778 (296h37m4.115555791s) [upload=332225]: Done uploading segments... Mar 18 19:37:06.518822033 (296h37m4.115593056s) [upload=332225]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 30.03242023s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 6 out of 60 is failed (threshold=6) [......................................................XXXXXX] duration: 5.001598488s trace: https://jaeger.dev.storj.io/trace/279181e60baf1770 FAIL (2026-03-18 19:36:52 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001626194s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 1m0.000802459s trace: https://jaeger.dev.storj.io/trace/704491ce75056034 FAIL (2026-03-18 19:36:50 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.00081902s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 1m0.030455742s trace: https://jaeger.dev.storj.io/trace/9413b9a7b24eb1b FAIL (2026-03-18 19:36:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== upload=332209]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 19:35:40.394401568 (296h35m37.991172604s) [upload=332209]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 19:35:40.422660789 (296h35m38.019431802s) [seg_pos={0 0}, upload=332209]: Begin upload segment... Mar 18 19:35:40.422731779 (296h35m38.019502804s) [seg_pos={0 0}, upload=332209]: Done begin upload segment. Mar 18 19:35:40.422774420 (296h35m38.019545451s) [noise=true , address=145.79.227.20:4460 , piece_id=KITCFIIN2I5TIK4HPTAJFNHIVRU3RRRGV6JG75HR6UBS5YXAPEIQ, seg_pos={0 0}, upload=332209]: Uploading piece... Mar 18 19:35:40.423159425 (296h35m38.019930459s) [noise=true , address=167.17.72.69:4731 , piece_id=H2QRYJTVLUKWSYSPLEJJJD5KFTN6IR4CVUK2DQEUS5MTLRRCOVGQ, seg_pos={0 0}, upload=332209]: Uploading piece... Mar 18 19:35:40.424057012 (296h35m38.020828035s) [noise=true , address=79.127.205.235:4261 , piece_id=JEAWP7LIMFRFIDLIER5WYNX5ZBC5LPFWTN4JGKBN7I57N54MA5BA, seg_pos={0 0}, upload=332209]: Uploading piece... Mar 18 19:35:40.424181031 (296h35m38.020952067s) [noise=true , address=45.91.116.8:4071 , piece_id=P72BBXIFWOBAWJMZHYUC6OAFY5XOQJRLMN7SOSQRXBFKZ62M47JA, seg_pos={0 0}, upload=332209]: Uploading piece... Mar 18 19:35:40.424585914 (296h35m38.021356952s) [noise=true , address=77.90.31.11:4580 , piece_id=LK6TGVC2Y5IE7LYJDES25NKKKACYZRTN3PNLNNPPU4WEQ3ZASGZQ, seg_pos={0 0}, upload=332209]: Uploading piece... Mar 18 19:35:40.424758856 (296h35m38.021529885s) [noise=true , address=193.149.164.71:4287 , piece_id=MR4GL6VQSQQWCHE7JPCZNROOEXXWSZBFO2HNSX753B6W7GNVNB5Q, seg_pos={0 0}, upload=332209]: Uploading piece... Mar 18 19:35:40.425025934 (296h35m38.021796968s) [noise=true , address=145.79.227.28:4756 , piece_id=WHO6CQNLVLYISWTCTTTBA7VEE4EB4THI56P5NO2JRLGIZPO6ADMA, seg_pos={0 0}, upload=332209]: Uploading piece... Mar 18 19:35:40.425285121 (296h35m38.022056145s) [noise=true , address=145.79.227.49:4350 , piece_id=I5PEKXKGDYGGRJ6XVXU64UA5TNYCEGH7NAUOXBD46XWZCBVKCJXQ, seg_pos={0 0}, upload=332209]: Uploading piece... Mar 18 19:35:40.425428344 (296h35m38.022199368s) [noise=true , address=79.127.205.237:4216 , piece_id=WWJC2764255DICP2IHUGM5HTIJBTUFGA64L64PIHEBYLSJQRJVDQ, seg_pos={0 0}, upload=332209]: Uploading piece... Mar 18 19:35:40.425795704 (296h35m38.022566727s) [noise=true , address=45.91.116.9:4282 , piece_id=LUO5ACVY34W6LZQXDBZBIOZZNDEMS253KN4LAEQ4OHEHK54BMSUA, seg_pos={0 0}, upload=332209]: Uploading piece... Mar 18 19:35:40.425842849 (296h35m38.022613872s) [noise=true , address=193.149.164.66:4990 , piece_id=ENU6NARWM56Z4H4EPJEJFW5BAIQNTA5ALAFAPQUZBYPCD7DW2WMA, seg_pos={0 0}, upload=332209]: Uploading piece... Mar 18 19:35:40.426270271 (296h35m38.023041294s) [noise=true , address=145.79.227.39:4047 , piece_id=NCAHP6WYUSDPZMGCVXXYIZWPWZPFFK3PXFZJAAUALLFH6DTCX5QA, seg_pos={0 0}, upload=332209]: Uploading piece... Mar 18 19:35:40.426386815 (296h35m38.023157846s) [noise=true , address=193.149.164.67:4011 , piece_id=QLHJBSADAQCDBP7XFMZTEZQGGVPG5X56TG2PLEGRTAUECK46AMRA, seg_pos={0 0}, upload=332209]: Uploading piece... Mar 18 19:35:40.426675310 (296h35m38.023446339s) [noise=true , address=77.90.31.19:4865 , piece_id=FTF6AKT7GZFMTCHFV3O7S4UVF5FJAOBBYP6MIDQYE2UDYLEMR6AA, seg_pos={0 0}, upload=332209]: Uploading piece... Mar 18 19:35:40.426962670 (296h35m38.023733704s) [noise=true , address=167.17.72.70:4748 , piece_id=VED3WQASFSWHNRUCCJTJJTDXPP3FKXA7VQ5HU3MI26SPUCO6EETA, seg_pos={0 0}, upload=332209]: Uploading piece... Mar 18 19:35:40.427581930 (296h35m38.024352953s) [noise=true , address=145.79.227.27:4702 , piece_id=GFMS5PJFLU362Q7CV5UUVONEKPPSUSSP2H4JORKX4ZV5WQGVYIUA, seg_pos={0 0}, upload=332209]: Uploading piece... Mar 18 19:35:40.430635108 (296h35m38.027406137s) [noise=true , address=79.127.219.46:4898 , piece_id=QHQIZLRJCVCV645RHQQAR5TPGHEJ6ZTSIZCXPOKF3M4YQ5A5XWMA, seg_pos={0 0}, upload=332209]: Uploading piece... Mar 18 19:35:40.431170788 (296h35m38.027941811s) [noise=true , address=145.79.227.40:4062 , piece_id=26MIMZ3WEBXQYJ5V5VK4MLG5FOSBAXGG2GRLUACG7VK6NZQXY3FA, seg_pos={0 0}, upload=332209]: Uploading piece... Mar 18 19:35:40.494145082 (296h35m38.090916108s) [upload=332209]: Next returned nil segment Mar 18 19:35:40.494162458 (296h35m38.090933484s) [upload=332209]: Waiting for error group managing segments... Mar 18 19:35:40.497483343 (296h35m38.094254367s) [noise=true , address=45.91.116.8:4071 , piece_id=P72BBXIFWOBAWJMZHYUC6OAFY5XOQJRLMN7SOSQRXBFKZ62M47JA, seg_pos={0 0}, upload=332209]: Done uploading piece. err: <nil> Mar 18 19:35:40.500470044 (296h35m38.097241068s) [noise=true , address=77.90.31.19:4865 , piece_id=FTF6AKT7GZFMTCHFV3O7S4UVF5FJAOBBYP6MIDQYE2UDYLEMR6AA, seg_pos={0 0}, upload=332209]: Done uploading piece. err: <nil> Mar 18 19:35:40.500529935 (296h35m38.09730096s) [noise=true , address=77.90.31.11:4580 , piece_id=LK6TGVC2Y5IE7LYJDES25NKKKACYZRTN3PNLNNPPU4WEQ3ZASGZQ, seg_pos={0 0}, upload=332209]: Done uploading piece. err: <nil> Mar 18 19:35:40.510898558 (296h35m38.107669584s) [noise=true , address=45.91.116.9:4282 , piece_id=LUO5ACVY34W6LZQXDBZBIOZZNDEMS253KN4LAEQ4OHEHK54BMSUA, seg_pos={0 0}, upload=332209]: Done uploading piece. err: <nil> Mar 18 19:35:40.528284266 (296h35m38.125055279s) [noise=true , address=79.127.219.46:4898 , piece_id=QHQIZLRJCVCV645RHQQAR5TPGHEJ6ZTSIZCXPOKF3M4YQ5A5XWMA, seg_pos={0 0}, upload=332209]: Done uploading piece. err: <nil> Mar 18 19:35:40.753353132 (296h35m38.350124156s) [noise=true , address=167.17.72.70:4748 , piece_id=VED3WQASFSWHNRUCCJTJJTDXPP3FKXA7VQ5HU3MI26SPUCO6EETA, seg_pos={0 0}, upload=332209]: Done uploading piece. err: <nil> Mar 18 19:35:40.754581752 (296h35m38.351352778s) [noise=true , address=167.17.72.69:4731 , piece_id=H2QRYJTVLUKWSYSPLEJJJD5KFTN6IR4CVUK2DQEUS5MTLRRCOVGQ, seg_pos={0 0}, upload=332209]: Done uploading piece. err: <nil> Mar 18 19:35:40.998337229 (296h35m38.595108256s) [noise=true , address=79.127.205.237:4216 , piece_id=WWJC2764255DICP2IHUGM5HTIJBTUFGA64L64PIHEBYLSJQRJVDQ, seg_pos={0 0}, upload=332209]: Done uploading piece. err: <nil> Mar 18 19:35:41.023461797 (296h35m38.62023282s) [noise=true , address=79.127.205.235:4261 , piece_id=JEAWP7LIMFRFIDLIER5WYNX5ZBC5LPFWTN4JGKBN7I57N54MA5BA, seg_pos={0 0}, upload=332209]: Done uploading piece. err: <nil> Mar 18 19:35:41.024571288 (296h35m38.621342321s) [noise=true , address=193.149.164.67:4011 , piece_id=QLHJBSADAQCDBP7XFMZTEZQGGVPG5X56TG2PLEGRTAUECK46AMRA, seg_pos={0 0}, upload=332209]: Done uploading piece. err: <nil> Mar 18 19:35:41.029079714 (296h35m38.625850738s) [noise=true , address=193.149.164.71:4287 , piece_id=MR4GL6VQSQQWCHE7JPCZNROOEXXWSZBFO2HNSX753B6W7GNVNB5Q, seg_pos={0 0}, upload=332209]: Done uploading piece. err: <nil> Mar 18 19:35:41.037731583 (296h35m38.634502611s) [noise=true , address=193.149.164.66:4990 , piece_id=ENU6NARWM56Z4H4EPJEJFW5BAIQNTA5ALAFAPQUZBYPCD7DW2WMA, seg_pos={0 0}, upload=332209]: Done uploading piece. err: <nil> Mar 18 19:36:40.365348172 (296h36m37.962119204s) [noise=true , address=145.79.227.28:4756 , piece_id=WHO6CQNLVLYISWTCTTTBA7VEE4EB4THI56P5NO2JRLGIZPO6ADMA, seg_pos={0 0}, upload=332209]: Done uploading piece. err: ecclient: upload failed (node:12pLAnfGYevL1h7SQBQ6yVM6CexBjC8EvHaj4Lvzj8aXGUfo1Hu, address:145.79.227.28:4756): protocol: rpc: dial tcp 145.79.227.28:4756: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:40.365444418 (296h36m37.962215444s) [noise=true , address=145.79.227.27:4702 , piece_id=GFMS5PJFLU362Q7CV5UUVONEKPPSUSSP2H4JORKX4ZV5WQGVYIUA, seg_pos={0 0}, upload=332209]: Done uploading piece. err: ecclient: upload failed (node:127aj2R9GJTXfH96zcHEVbcsTWMT52HWiCrLZZ8c6tZsxDe1xrj, address:145.79.227.27:4702): protocol: rpc: dial tcp 145.79.227.27:4702: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:40.365575720 (296h36m37.96234684s) [noise=true , address=145.79.227.49:4350 , piece_id=I5PEKXKGDYGGRJ6XVXU64UA5TNYCEGH7NAUOXBD46XWZCBVKCJXQ, seg_pos={0 0}, upload=332209]: Done uploading piece. err: ecclient: upload failed (node:12byqLui89KiVQB8FetY1aAZAX3Ziy6orZr4DeSgLFjpFQe2CH, address:145.79.227.49:4350): protocol: rpc: dial tcp 145.79.227.49:4350: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:40.366042812 (296h36m37.962813847s) [noise=true , address=145.79.227.39:4047 , piece_id=NCAHP6WYUSDPZMGCVXXYIZWPWZPFFK3PXFZJAAUALLFH6DTCX5QA, seg_pos={0 0}, upload=332209]: Done uploading piece. err: ecclient: upload failed (node:126Z6tvp49dLLdj3T5snJc4tBVGzcX5UBaGafFN3PkhF92vxpfs, address:145.79.227.39:4047): protocol: rpc: dial tcp 145.79.227.39:4047: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:40.366528812 (296h36m37.963299845s) [noise=true , address=145.79.227.20:4460 , piece_id=KITCFIIN2I5TIK4HPTAJFNHIVRU3RRRGV6JG75HR6UBS5YXAPEIQ, seg_pos={0 0}, upload=332209]: Done uploading piece. err: ecclient: upload failed (node:12nWW84ncAoTpCJjUpKuTWLVZarxy4jMZAB3RSrFZ5skxycKtqh, address:145.79.227.20:4460): protocol: rpc: dial tcp 145.79.227.20:4460: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Mar 18 19:36:40.366635690 (296h36m37.963406734s) [noise=true , address=145.79.227.40:4062 , piece_id=26MIMZ3WEBXQYJ5V5VK4MLG5FOSBAXGG2GRLUACG7VK6NZQXY3FA, seg_pos={0 0}, upload=332209]: Done uploading piece. err: ecclient: upload failed (node:12N61Va6p4WU9cmUz65CkCDKr54LFtJhh8HJGYPVt8cJTRoWLhb, address:145.79.227.40:4062): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:36:40.366693261 (296h36m37.963464286s) [seg_pos={0 0}, upload=332209]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Mar 18 19:36:40.392095339 (296h36m37.988866352s) [upload=332209]: Done uploading segments... Mar 18 19:36:40.392147141 (296h36m37.988918164s) [upload=332209]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 1m0.030484518s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................................................XXXX] duration: 30.02496055s trace: https://jaeger.dev.storj.io/trace/101304de33264c0b FAIL (2026-03-18 19:36:35 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== stItem_ObjectBegin Mar 18 19:36:05.779298320 (296h36m3.376069355s) [upload=332216]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 19:36:05.806291505 (296h36m3.403062559s) [seg_pos={0 0}, upload=332216]: Begin upload segment... Mar 18 19:36:05.806360085 (296h36m3.40313112s) [seg_pos={0 0}, upload=332216]: Done begin upload segment. Mar 18 19:36:05.806426935 (296h36m3.403197968s) [noise=true , address=145.79.227.40:4052 , piece_id=477W7TS5VNJW4MMM5XYT4ROFPW634G6KRQV5MV3TV2DMZOVINVTQ, seg_pos={0 0}, upload=332216]: Uploading piece... Mar 18 19:36:05.806434620 (296h36m3.403205629s) [noise=true , address=45.91.116.12:4227 , piece_id=FKODVRBOF4E7SKNPJUPURQ4IWFHWUJ5UCK5ROQ5QV5YIGCTZYOAA, seg_pos={0 0}, upload=332216]: Uploading piece... Mar 18 19:36:05.807110168 (296h36m3.403881201s) [noise=true , address=145.79.227.46:4606 , piece_id=VACSSCON2SHL5WPKMPM3HHAP5A2CQJTLL6LXLG6CK3EYESUIXJFQ, seg_pos={0 0}, upload=332216]: Uploading piece... Mar 18 19:36:05.807270695 (296h36m3.404042223s) [noise=true , address=45.91.116.11:4152 , piece_id=52AP37654RCD3MTYF2MIO4L7M3A5PV662QVUCQLUQ64SRO7J4MFA, seg_pos={0 0}, upload=332216]: Uploading piece... Mar 18 19:36:05.807900551 (296h36m3.404671577s) [noise=true , address=145.79.227.38:4980 , piece_id=RN3DE7JSUIXXNF3A52ZRAXOWBFLY2OSPY7MODGQZUIYYHIWPTJPQ, seg_pos={0 0}, upload=332216]: Uploading piece... Mar 18 19:36:05.807911079 (296h36m3.404682112s) [noise=true , address=79.127.219.46:4892 , piece_id=KSB6YMJMR33PTYBVV36EIUBHTHJOTM4QR2FHJHF5O53YFJGBSZCQ, seg_pos={0 0}, upload=332216]: Uploading piece... Mar 18 19:36:05.808357278 (296h36m3.405128312s) [noise=true , address=45.91.116.4:4937 , piece_id=RGJU5D6RUMGTPAXY53JBS4OLNIPHM3Q3DJVFAWW7DZZHJEPV6CZA, seg_pos={0 0}, upload=332216]: Uploading piece... Mar 18 19:36:05.808468201 (296h36m3.405239235s) [noise=true , address=45.91.116.8:4070 , piece_id=AERIDKRDXUP6AF3EDLPZTEDZMBKMIB3A5SAUGBKTQVADGOJGWFEQ, seg_pos={0 0}, upload=332216]: Uploading piece... Mar 18 19:36:05.808830565 (296h36m3.405601598s) [noise=true , address=145.79.227.28:4730 , piece_id=MPEA32QBV5UQTGOL25RIFFC7ZXV3FOVV7LNM535QZMUQC565GYQQ, seg_pos={0 0}, upload=332216]: Uploading piece... Mar 18 19:36:05.809038567 (296h36m3.4058096s) [noise=true , address=145.79.227.17:4974 , piece_id=B46W2TXOZ54IP2VZMWS7GD75VYG2TXY5UMKBU4RNOMHNI6TB3P3A, seg_pos={0 0}, upload=332216]: Uploading piece... Mar 18 19:36:05.809276307 (296h36m3.406047343s) [noise=true , address=145.79.227.14:4840 , piece_id=TTZMYGRALHWBHYO27GEBVT755QNG5WAHNVPFUKAHCLCPCJEYTBMA, seg_pos={0 0}, upload=332216]: Uploading piece... Mar 18 19:36:05.809604647 (296h36m3.406375678s) [noise=true , address=145.79.227.7:4297 , piece_id=WDKKRXV5VOMIVH5PJ34FEKGU2AKFGSIMKHNZVXERPEL24ND5HDYA, seg_pos={0 0}, upload=332216]: Uploading piece... Mar 18 19:36:05.809732811 (296h36m3.406503843s) [noise=true , address=145.79.227.8:4332 , piece_id=HOLTDLM7OUYWDJS4HGEA4KTPNXGPSYHYT3Y5U6TNOLSMJY77ZV7Q, seg_pos={0 0}, upload=332216]: Uploading piece... Mar 18 19:36:05.810215139 (296h36m3.406986171s) [noise=true , address=145.79.227.29:4942 , piece_id=NRL4DUTPBBHTARZLHGM27RWRZMQAEFB3ZDI4UJPA6Z7AWHBZJO4Q, seg_pos={0 0}, upload=332216]: Uploading piece... Mar 18 19:36:05.810920587 (296h36m3.407691622s) [noise=true , address=145.79.227.13:4807 , piece_id=EMJOYUZU7B2OUQWN757AEBYUZY6WL5NVTF35URBUDPR3ZYFISVKA, seg_pos={0 0}, upload=332216]: Uploading piece... Mar 18 19:36:05.811707365 (296h36m3.408478399s) [noise=true , address=77.90.31.21:4882 , piece_id=YOA5RWMFUKH5MHPMTQC2NDAGPXEOL5L4XARX3W3DL7Q7XOIPG2RA, seg_pos={0 0}, upload=332216]: Uploading piece... Mar 18 19:36:05.882828632 (296h36m3.479599648s) [upload=332216]: Next returned nil segment Mar 18 19:36:05.882843872 (296h36m3.479614896s) [upload=332216]: Waiting for error group managing segments... Mar 18 19:36:05.892421543 (296h36m3.489192567s) [noise=true , address=79.127.219.46:4892 , piece_id=KSB6YMJMR33PTYBVV36EIUBHTHJOTM4QR2FHJHF5O53YFJGBSZCQ, seg_pos={0 0}, upload=332216]: Done uploading piece. err: <nil> Mar 18 19:36:05.892642623 (296h36m3.489413639s) [noise=true , address=77.90.31.21:4882 , piece_id=YOA5RWMFUKH5MHPMTQC2NDAGPXEOL5L4XARX3W3DL7Q7XOIPG2RA, seg_pos={0 0}, upload=332216]: Done uploading piece. err: <nil> Mar 18 19:36:05.892991134 (296h36m3.489762161s) [noise=true , address=45.91.116.8:4070 , piece_id=AERIDKRDXUP6AF3EDLPZTEDZMBKMIB3A5SAUGBKTQVADGOJGWFEQ, seg_pos={0 0}, upload=332216]: Done uploading piece. err: <nil> Mar 18 19:36:05.893075585 (296h36m3.489846597s) [noise=true , address=45.91.116.4:4937 , piece_id=RGJU5D6RUMGTPAXY53JBS4OLNIPHM3Q3DJVFAWW7DZZHJEPV6CZA, seg_pos={0 0}, upload=332216]: Done uploading piece. err: <nil> Mar 18 19:36:05.912318365 (296h36m3.50908941s) [noise=true , address=45.91.116.11:4152 , piece_id=52AP37654RCD3MTYF2MIO4L7M3A5PV662QVUCQLUQ64SRO7J4MFA, seg_pos={0 0}, upload=332216]: Done uploading piece. err: <nil> Mar 18 19:36:05.912708313 (296h36m3.50947934s) [noise=true , address=45.91.116.12:4227 , piece_id=FKODVRBOF4E7SKNPJUPURQ4IWFHWUJ5UCK5ROQ5QV5YIGCTZYOAA, seg_pos={0 0}, upload=332216]: Done uploading piece. err: <nil> Mar 18 19:36:35.745332183 (296h36m33.342103487s) [noise=true , address=145.79.227.46:4606 , piece_id=VACSSCON2SHL5WPKMPM3HHAP5A2CQJTLL6LXLG6CK3EYESUIXJFQ, seg_pos={0 0}, upload=332216]: Done uploading piece. err: ecclient: upload failed (node:12SMRAQn7BaX1Fzpvou2JHaiLko9fcQLdYggaVcjRihU516CbRh, address:145.79.227.46:4606): protocol: rpc: dial tcp 145.79.227.46:4606: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:35.745705612 (296h36m33.342476645s) [noise=true , address=145.79.227.7:4297 , piece_id=WDKKRXV5VOMIVH5PJ34FEKGU2AKFGSIMKHNZVXERPEL24ND5HDYA, seg_pos={0 0}, upload=332216]: Done uploading piece. err: ecclient: upload failed (node:1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, address:145.79.227.7:4297): protocol: rpc: dial tcp 145.79.227.7:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:35.746030571 (296h36m33.342801605s) [noise=true , address=145.79.227.40:4052 , piece_id=477W7TS5VNJW4MMM5XYT4ROFPW634G6KRQV5MV3TV2DMZOVINVTQ, seg_pos={0 0}, upload=332216]: Done uploading piece. err: ecclient: upload failed (node:12PViJBYAanwXRWhMq3WHR6k9Stxy8hjuY5FV9J7ErxCy4N1q3V, address:145.79.227.40:4052): protocol: rpc: dial tcp 145.79.227.40:4052: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:35.746400847 (296h36m33.34317188s) [noise=true , address=145.79.227.38:4980 , piece_id=RN3DE7JSUIXXNF3A52ZRAXOWBFLY2OSPY7MODGQZUIYYHIWPTJPQ, seg_pos={0 0}, upload=332216]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: rpc: dial tcp 145.79.227.38:4980: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:35.746761111 (296h36m33.343532143s) [noise=true , address=145.79.227.17:4974 , piece_id=B46W2TXOZ54IP2VZMWS7GD75VYG2TXY5UMKBU4RNOMHNI6TB3P3A, seg_pos={0 0}, upload=332216]: Done uploading piece. err: ecclient: upload failed (node:1aR2XEwckLo7ufnCtsmyG2PS3vNiqpZs9b66UhuH7vUixcePS1, address:145.79.227.17:4974): protocol: rpc: dial tcp 145.79.227.17:4974: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:35.747303389 (296h36m33.344074421s) [noise=true , address=145.79.227.14:4840 , piece_id=TTZMYGRALHWBHYO27GEBVT755QNG5WAHNVPFUKAHCLCPCJEYTBMA, seg_pos={0 0}, upload=332216]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: rpc: dial tcp 145.79.227.14:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:35.747922968 (296h36m33.344694s) [noise=true , address=145.79.227.29:4942 , piece_id=NRL4DUTPBBHTARZLHGM27RWRZMQAEFB3ZDI4UJPA6Z7AWHBZJO4Q, seg_pos={0 0}, upload=332216]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: rpc: dial tcp 145.79.227.29:4942: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:35.748024876 (296h36m33.344795909s) [noise=true , address=145.79.227.13:4807 , piece_id=EMJOYUZU7B2OUQWN757AEBYUZY6WL5NVTF35URBUDPR3ZYFISVKA, seg_pos={0 0}, upload=332216]: Done uploading piece. err: ecclient: upload failed (node:1MZ9LJLHF4XXvmRG2r7XkanDfdNSKCUgua9EBPusPuGyLzTLo8, address:145.79.227.13:4807): protocol: rpc: dial tcp 145.79.227.13:4807: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:35.748385105 (296h36m33.345156141s) [noise=true , address=145.79.227.28:4730 , piece_id=MPEA32QBV5UQTGOL25RIFFC7ZXV3FOVV7LNM535QZMUQC565GYQQ, seg_pos={0 0}, upload=332216]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: rpc: dial tcp 145.79.227.28:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:35.748666388 (296h36m33.345437424s) [noise=true , address=145.79.227.8:4332 , piece_id=HOLTDLM7OUYWDJS4HGEA4KTPNXGPSYHYT3Y5U6TNOLSMJY77ZV7Q, seg_pos={0 0}, upload=332216]: Done uploading piece. err: ecclient: upload failed (node:1khNvxTmr2av1PFmxCv4cEcMv25F9y41bndYPQKuP2LCx7WoMQ, address:145.79.227.8:4332): protocol: rpc: dial tcp 145.79.227.8:4332: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:35.748710372 (296h36m33.345481399s) [seg_pos={0 0}, upload=332216]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Mar 18 19:36:35.768776373 (296h36m33.365547404s) [upload=332216]: Done uploading segments... Mar 18 19:36:35.768816414 (296h36m33.36558744s) [upload=332216]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration:* 30.024979911s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 207.158221ms trace: https://jaeger.dev.storj.io/trace/4e61caa8af497b0f FAIL (2026-03-18 19:36:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: internal error ======LOGS====== Mar 18 18:50:37.197689801 (295h50m34.794460832s) [upload=331359]: Starting upload Mar 18 18:50:37.197768007 (295h50m34.79453904s) [upload=331359]: Done starting upload Mar 18 18:50:37.197865441 (295h50m34.79463646s) [upload=331359]: Uploading segments... Mar 18 18:50:37.198001255 (295h50m34.794772285s) [upload=331359]: Got next segment. Inline: true Mar 18 18:50:37.198009821 (295h50m34.794780854s) [upload=331359]: Waiting for error group managing segments... Mar 18 18:50:37.198092531 (295h50m34.794863559s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 18:50:37.198099572 (295h50m34.794870604s) [upload=331359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 18:50:37.198103547 (295h50m34.794874571s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 18:50:37.234689079 (295h50m34.831460108s) [upload=331359]: Done uploading segments... Mar 18 18:50:37.234743231 (295h50m34.831514254s) [upload=331359]: Upload finished. err: metaclient: internal error Duration:* 207.188732ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [.......................................................XXXXX] duration: 5.002146604s trace: https://jaeger.dev.storj.io/trace/493063e607864501 FAIL (2026-03-18 19:36:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.002175892s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 30.024294442s trace: https://jaeger.dev.storj.io/trace/629f540958e18e1d FAIL (2026-03-18 19:36:05 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== 4:4190 , piece_id=XCV66HQXWIL3PQXFITZAGLXOJPPA7CTVVM7CANDTQLUKWMKM2PHA, seg_pos={0 0}, upload=332208]: Uploading piece... Mar 18 19:35:35.260074257 (296h35m32.856845291s) [noise=true , address=145.79.227.26:4656 , piece_id=D7VE57XGSMI377C4IFI72EXJYYQACBU2KYLK274E6KSAKI2RV2QQ, seg_pos={0 0}, upload=332208]: Uploading piece... Mar 18 19:35:35.263237994 (296h35m32.860009028s) [noise=true , address=145.79.227.13:4806 , piece_id=GRC4ZY74JHDAU6JTPIUY46CN5SD4GFQW7A3BJD7K2LMIV2T6ZIWQ, seg_pos={0 0}, upload=332208]: Uploading piece... Mar 18 19:35:35.263533433 (296h35m32.860304466s) [noise=true , address=45.91.116.9:4261 , piece_id=7AJV3XT4X4TY7KZXLG6Q5NORKFIKW5DNBLWAHWLER4XQMSDMMQOQ, seg_pos={0 0}, upload=332208]: Uploading piece... Mar 18 19:35:35.264101470 (296h35m32.860872501s) [noise=true , address=145.79.227.28:4728 , piece_id=Z4MSAXBHANYBONUZCYIDE3OSCIPCRM4GYUYXVZ5N4J7LDLBU2BEA, seg_pos={0 0}, upload=332208]: Uploading piece... Mar 18 19:35:35.264159838 (296h35m32.860930863s) [noise=true , address=145.79.227.11:4733 , piece_id=H33OYHHQQLL532O2COEDTGOC5KBICO7FJVONQCN3RRTH6XWS4TMQ, seg_pos={0 0}, upload=332208]: Uploading piece... Mar 18 19:35:35.264840424 (296h35m32.861611458s) [noise=true , address=45.91.116.6:4975 , piece_id=NUR3KSS6H2IN62MHHVKM4VNDBFZQUQ45YECZIVBGGY7NB2P5UINA, seg_pos={0 0}, upload=332208]: Uploading piece... Mar 18 19:35:35.265481653 (296h35m32.862252677s) [noise=true , address=145.79.227.48:4301 , piece_id=PSODOSW7IS4NSO7QMPCMXSMBHUZU3HPDBRFXIWQEKOQRHEVVHPGA, seg_pos={0 0}, upload=332208]: Uploading piece... Mar 18 19:35:35.266118632 (296h35m32.862889666s) [noise=true , address=77.90.31.12:4590 , piece_id=DENYDULPV2HWKPPJ2I5TVUZTQO7TXDAWYQD7E4OFEZ7KPBZKSK3Q, seg_pos={0 0}, upload=332208]: Uploading piece... Mar 18 19:35:35.269859923 (296h35m32.866630949s) [noise=true , address=145.79.227.40:4049 , piece_id=UUMEI6ALPBC5GVVQS45SLPBC2GFZT3Z25PQ2VA6L5ZGRSSINGFOA, seg_pos={0 0}, upload=332208]: Uploading piece... Mar 18 19:35:35.282434918 (296h35m32.879205946s) [noise=true , address=145.79.227.29:4942 , piece_id=LLC3P536L24QQX66FIVDYUQ6S2UX67THGYGJKQMFZ7H2465VAKFA, seg_pos={0 0}, upload=332208]: Uploading piece... Mar 18 19:35:35.286107324 (296h35m32.882878356s) [noise=true , address=145.79.227.18:4011 , piece_id=SVO43CNSUEWPWW5THMQBEFXRTRJYHROAPO37M6ZMN6IO3SVBRBUQ, seg_pos={0 0}, upload=332208]: Uploading piece... Mar 18 19:35:35.289452404 (296h35m32.886223443s) [noise=true , address=145.79.227.22:4512 , piece_id=HL3AY4QPXBZVCW4DWJLSPPCF5ISRAW3L3G5OIKDJIS4RI3THNC5Q, seg_pos={0 0}, upload=332208]: Uploading piece... Mar 18 19:35:35.292646433 (296h35m32.889417466s) [noise=true , address=145.79.227.35:4483 , piece_id=575UNXTOJIARWIB5ZLTOUBAPFE7KI4FYLE7EQMH7QS5KOLHRXTRA, seg_pos={0 0}, upload=332208]: Uploading piece... Mar 18 19:35:35.294680659 (296h35m32.891451688s) [noise=true , address=145.79.227.2:4115 , piece_id=VZSIJ6V6V32FDQDNZMPPBWUPVQD46OPI5N2ZP5D73IV6WUXZ5GTA, seg_pos={0 0}, upload=332208]: Uploading piece... Mar 18 19:35:35.338109701 (296h35m32.934880713s) [upload=332208]: Next returned nil segment Mar 18 19:35:35.338122831 (296h35m32.934893844s) [upload=332208]: Waiting for error group managing segments... Mar 18 19:35:35.341950953 (296h35m32.938721977s) [noise=true , address=45.91.116.9:4261 , piece_id=7AJV3XT4X4TY7KZXLG6Q5NORKFIKW5DNBLWAHWLER4XQMSDMMQOQ, seg_pos={0 0}, upload=332208]: Done uploading piece. err: <nil> Mar 18 19:35:35.342668191 (296h35m32.939439204s) [noise=true , address=45.91.116.6:4975 , piece_id=NUR3KSS6H2IN62MHHVKM4VNDBFZQUQ45YECZIVBGGY7NB2P5UINA, seg_pos={0 0}, upload=332208]: Done uploading piece. err: <nil> Mar 18 19:35:35.343934994 (296h35m32.940706018s) [noise=true , address=77.90.31.12:4590 , piece_id=DENYDULPV2HWKPPJ2I5TVUZTQO7TXDAWYQD7E4OFEZ7KPBZKSK3Q, seg_pos={0 0}, upload=332208]: Done uploading piece. err: <nil> Mar 18 19:36:05.199875509 (296h36m2.796646533s) [noise=true , address=145.79.227.11:4733 , piece_id=H33OYHHQQLL532O2COEDTGOC5KBICO7FJVONQCN3RRTH6XWS4TMQ, seg_pos={0 0}, upload=332208]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:05.199968555 (296h36m2.7967396s) [noise=true , address=145.79.227.47:4268 , piece_id=J22U26GR3KXCTM223HYEGGVILVRJWQYH4S3UZA6TMCP2EGEYXJDA, seg_pos={0 0}, upload=332208]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: rpc: dial tcp 145.79.227.47:4268: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:05.200058227 (296h36m2.796829261s) [noise=true , address=145.79.227.4:4190 , piece_id=XCV66HQXWIL3PQXFITZAGLXOJPPA7CTVVM7CANDTQLUKWMKM2PHA, seg_pos={0 0}, upload=332208]: Done uploading piece. err: ecclient: upload failed (node:1eNJzs5Dc7Nv7imoQXZDp2mEVWjD8o4viGVkFvuAeZ1arynxu4, address:145.79.227.4:4190): protocol: rpc: dial tcp 145.79.227.4:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:05.200188018 (296h36m2.796959041s) [noise=true , address=145.79.227.26:4656 , piece_id=D7VE57XGSMI377C4IFI72EXJYYQACBU2KYLK274E6KSAKI2RV2QQ, seg_pos={0 0}, upload=332208]: Done uploading piece. err: ecclient: upload failed (node:1u6e2ZjrPg2q3EDN2ZjDj9hzEmCAoRvt39HMeM79gcVmMVm53j, address:145.79.227.26:4656): protocol: rpc: dial tcp 145.79.227.26:4656: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:05.200225730 (296h36m2.796996766s) [noise=true , address=145.79.227.48:4301 , piece_id=PSODOSW7IS4NSO7QMPCMXSMBHUZU3HPDBRFXIWQEKOQRHEVVHPGA, seg_pos={0 0}, upload=332208]: Done uploading piece. err: ecclient: upload failed (node:18HkaSrdHHnvmeNz1ormfcSBZH8bMUgkX7X3JmGuqkmtBoak29, address:145.79.227.48:4301): protocol: rpc: dial tcp 145.79.227.48:4301: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:05.200242062 (296h36m2.797013091s) [noise=true , address=145.79.227.40:4049 , piece_id=UUMEI6ALPBC5GVVQS45SLPBC2GFZT3Z25PQ2VA6L5ZGRSSINGFOA, seg_pos={0 0}, upload=332208]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: rpc: dial tcp 145.79.227.40:4049: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:05.200296255 (296h36m2.797067281s) [noise=true , address=145.79.227.13:4806 , piece_id=GRC4ZY74JHDAU6JTPIUY46CN5SD4GFQW7A3BJD7K2LMIV2T6ZIWQ, seg_pos={0 0}, upload=332208]: Done uploading piece. err: ecclient: upload failed (node:12qUyweKQhw15RjewCgKi1oLyCvN3VUB1VMpeGhj2Q2hszvJBsQ, address:145.79.227.13:4806): protocol: rpc: dial tcp 145.79.227.13:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:05.200540599 (296h36m2.797311636s) [noise=true , address=145.79.227.29:4942 , piece_id=LLC3P536L24QQX66FIVDYUQ6S2UX67THGYGJKQMFZ7H2465VAKFA, seg_pos={0 0}, upload=332208]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: rpc: dial tcp 145.79.227.29:4942: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:05.200684225 (296h36m2.797455253s) [noise=true , address=145.79.227.35:4483 , piece_id=575UNXTOJIARWIB5ZLTOUBAPFE7KI4FYLE7EQMH7QS5KOLHRXTRA, seg_pos={0 0}, upload=332208]: Done uploading piece. err: ecclient: upload failed (node:1igSs2ZGX3zWVsSdmm1pnwYd7JckcyBubTuNU1gJDxQsZ9nKaY, address:145.79.227.35:4483): protocol: rpc: dial tcp 145.79.227.35:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:05.200782705 (296h36m2.797553739s) [noise=true , address=145.79.227.22:4512 , piece_id=HL3AY4QPXBZVCW4DWJLSPPCF5ISRAW3L3G5OIKDJIS4RI3THNC5Q, seg_pos={0 0}, upload=332208]: Done uploading piece. err: ecclient: upload failed (node:12cnVqYYfo8b6RvRtMykzh8mU7UAqxcqYvWe7H8G7k8DqtNLAR4, address:145.79.227.22:4512): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:36:05.200974989 (296h36m2.797746015s) [noise=true , address=145.79.227.2:4115 , piece_id=VZSIJ6V6V32FDQDNZMPPBWUPVQD46OPI5N2ZP5D73IV6WUXZ5GTA, seg_pos={0 0}, upload=332208]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: rpc: dial tcp 145.79.227.2:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:05.201267097 (296h36m2.798038126s) [noise=true , address=145.79.227.28:4728 , piece_id=Z4MSAXBHANYBONUZCYIDE3OSCIPCRM4GYUYXVZ5N4J7LDLBU2BEA, seg_pos={0 0}, upload=332208]: Done uploading piece. err: ecclient: upload failed (node:1YoLzpb4mMYZ8TsdxnMocekZZ1gLfTrdCM8mxTDj4tY6khZGvi, address:145.79.227.28:4728): protocol: rpc: dial tcp 145.79.227.28:4728: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:05.201623799 (296h36m2.798394826s) [noise=true , address=145.79.227.18:4011 , piece_id=SVO43CNSUEWPWW5THMQBEFXRTRJYHROAPO37M6ZMN6IO3SVBRBUQ, seg_pos={0 0}, upload=332208]: Done uploading piece. err: ecclient: upload failed (node:12J8J47emDh4p115m69EqUYroi9GgruC3y8gmv2xVULVJSvFYYP, address:145.79.227.18:4011): protocol: rpc: dial tcp 145.79.227.18:4011: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:36:05.201658060 (296h36m2.798429092s) [seg_pos={0 0}, upload=332208]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Mar 18 19:36:05.221098688 (296h36m2.81786972s) [upload=332208]: Done uploading segments... Mar 18 19:36:05.221146194 (296h36m2.817917221s) [upload=332208]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 30.024337517s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 1m0.001389534s trace: https://jaeger.dev.storj.io/trace/37d9b9a8822ff571 FAIL (2026-03-18 19:35:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.00142373s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.038034506s trace: https://jaeger.dev.storj.io/trace/3bdb4ca70f3434e4 FAIL (2026-03-18 19:35:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== segment. Inline: false Mar 18 19:34:40.288697242 (296h34m37.885468273s) [upload=332190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 19:34:40.288704567 (296h34m37.885475601s) [upload=332190]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 19:34:40.316240026 (296h34m37.913011052s) [seg_pos={0 0}, upload=332190]: Begin upload segment... Mar 18 19:34:40.316461452 (296h34m37.913232466s) [noise=true , address=79.127.205.235:4282 , piece_id=UJIZ3MRUFLTSHSUUVGXHPZB66YGYT6NJLI4N32BKLDI6A6HWOBAA, seg_pos={0 0}, upload=332190]: Uploading piece... Mar 18 19:34:40.317429625 (296h34m37.914200648s) [seg_pos={0 0}, upload=332190]: Done begin upload segment. Mar 18 19:34:40.317458909 (296h34m37.914229938s) [noise=true , address=193.149.164.71:4298 , piece_id=ZTLYCQGH65XSOWC2ZDZFMXSYB6SKGCX5DS7SVWKDMO3CWB6CQDSA, seg_pos={0 0}, upload=332190]: Uploading piece... Mar 18 19:34:40.317475751 (296h34m37.914246775s) [noise=true , address=77.90.31.16:4753 , piece_id=6H2CQVMNGMMODIJXTXDUEEG2UQ5GIKOWJE5QLZDUC6RXXGLSTRMA, seg_pos={0 0}, upload=332190]: Uploading piece... Mar 18 19:34:40.318029471 (296h34m37.914800497s) [noise=true , address=79.127.219.46:4905 , piece_id=GECZVFQ26W2GHC5NFFB5X4S7KFK7LWC5NEGW2ZUAUH6MJ23ZSD3A, seg_pos={0 0}, upload=332190]: Uploading piece... Mar 18 19:34:40.318614705 (296h34m37.915385727s) [noise=true , address=193.149.164.69:4082 , piece_id=B4WBGJ4DDHDF7LSA6XNLROPT5KR76BTHBXPTYYZOMU24NUREQUTA, seg_pos={0 0}, upload=332190]: Uploading piece... Mar 18 19:34:40.319600202 (296h34m37.916371226s) [noise=true , address=145.79.227.52:4451 , piece_id=7I6S4NZHN5GIBWSKXIGJJ2BWUSRRWO534EHLVHEWRWP6K5CWBVHA, seg_pos={0 0}, upload=332190]: Uploading piece... Mar 18 19:34:40.320113526 (296h34m37.91688455s) [noise=true , address=145.79.227.12:4791 , piece_id=2OTNUFAPZAX6VQFII6OQFT2BEAUS4PBJC36XN7FBO7OO2IF3QGNA, seg_pos={0 0}, upload=332190]: Uploading piece... Mar 18 19:34:40.322192667 (296h34m37.918963696s) [noise=true , address=193.149.164.66:4995 , piece_id=B4N4RTC3AUNDI32SOGFNCLAXOYS4JVKT7GCTRCYBHYFGHX77RL3Q, seg_pos={0 0}, upload=332190]: Uploading piece... Mar 18 19:34:40.323252463 (296h34m37.920023486s) [noise=true , address=45.91.116.4:4959 , piece_id=LASRRSQHDDQ2BJSSS6GA673NS55ZHA7JO2PUHT2AWJSX73AW7NKA, seg_pos={0 0}, upload=332190]: Uploading piece... Mar 18 19:34:40.326062420 (296h34m37.922833447s) [noise=true , address=193.149.164.70:4110 , piece_id=X24XJHJHWP6LWFML5T7X2IIV7V7OHOQ2K4CZGO7BLYQAGMEJPESQ, seg_pos={0 0}, upload=332190]: Uploading piece... Mar 18 19:34:40.326970828 (296h34m37.923741846s) [noise=true , address=145.79.227.13:4835 , piece_id=SZUFWGRLFYV3YB2KTVDPPZN23PBUHQD5LI32LDYHSKUWB3URSYBQ, seg_pos={0 0}, upload=332190]: Uploading piece... Mar 18 19:34:40.327948307 (296h34m37.924719324s) [noise=true , address=145.79.227.26:4665 , piece_id=QLOE3K4G5XTGDTAZP4APCFVXO2QV57TU7HCZC6KB7D42D2WYXEMQ, seg_pos={0 0}, upload=332190]: Uploading piece... Mar 18 19:34:40.331330150 (296h34m37.928101167s) [noise=true , address=79.127.219.45:4859 , piece_id=T57ANTKHSAFA6PPG6DVLBF3EDR45WGE4CCBP2MLV3NCSPEVMTTXA, seg_pos={0 0}, upload=332190]: Uploading piece... Mar 18 19:34:40.333935669 (296h34m37.930706687s) [noise=true , address=145.79.227.4:4203 , piece_id=BFRKIWDBFBOXGYSEPEUHTI24YQ6KNFQ6NU5WDWWQQ44KD4CGT4LQ, seg_pos={0 0}, upload=332190]: Uploading piece... Mar 18 19:34:40.336210475 (296h34m37.932981498s) [noise=true , address=79.127.219.38:4840 , piece_id=RLBSG7OCFDQ4FM4HKCYWNX7UFYK24UYAIHEHZ4Y4WUVEL5LQKJ5A, seg_pos={0 0}, upload=332190]: Uploading piece... Mar 18 19:34:40.338501139 (296h34m37.935272162s) [noise=true , address=45.91.116.10:4107 , piece_id=JOUZQ2R7QHSRH6QD6ZFYN6AYJ4VBUNOK6SHBJNLGYY4YHODLZLAA, seg_pos={0 0}, upload=332190]: Uploading piece... Mar 18 19:34:40.340573090 (296h34m37.937344112s) [noise=true , address=145.79.227.45:4235 , piece_id=FGQMFHRGSP7RZZLN4QAYUTQRP4U6H5T3FCOKVBTA7UBXG4KVKUUQ, seg_pos={0 0}, upload=332190]: Uploading piece... Mar 18 19:34:40.342729921 (296h34m37.93950096s) [noise=true , address=77.90.31.13:4647 , piece_id=2PVNNE6NKKZYIZH3ACI73OSOXA7P6WC24VIY22TOCHUQTYCY5Q6Q, seg_pos={0 0}, upload=332190]: Uploading piece... Mar 18 19:34:40.393958171 (296h34m37.990729204s) [upload=332190]: Next returned nil segment Mar 18 19:34:40.393976093 (296h34m37.990747126s) [upload=332190]: Waiting for error group managing segments... Mar 18 19:34:40.398166434 (296h34m37.994937457s) [noise=true , address=79.127.219.38:4840 , piece_id=RLBSG7OCFDQ4FM4HKCYWNX7UFYK24UYAIHEHZ4Y4WUVEL5LQKJ5A, seg_pos={0 0}, upload=332190]: Done uploading piece. err: <nil> Mar 18 19:34:40.413881432 (296h34m38.010652447s) [noise=true , address=77.90.31.13:4647 , piece_id=2PVNNE6NKKZYIZH3ACI73OSOXA7P6WC24VIY22TOCHUQTYCY5Q6Q, seg_pos={0 0}, upload=332190]: Done uploading piece. err: <nil> Mar 18 19:34:40.414281127 (296h34m38.011052153s) [noise=true , address=45.91.116.10:4107 , piece_id=JOUZQ2R7QHSRH6QD6ZFYN6AYJ4VBUNOK6SHBJNLGYY4YHODLZLAA, seg_pos={0 0}, upload=332190]: Done uploading piece. err: <nil> Mar 18 19:34:40.415850088 (296h34m38.012621113s) [noise=true , address=79.127.219.45:4859 , piece_id=T57ANTKHSAFA6PPG6DVLBF3EDR45WGE4CCBP2MLV3NCSPEVMTTXA, seg_pos={0 0}, upload=332190]: Done uploading piece. err: <nil> Mar 18 19:34:40.417360136 (296h34m38.014131162s) [noise=true , address=45.91.116.4:4959 , piece_id=LASRRSQHDDQ2BJSSS6GA673NS55ZHA7JO2PUHT2AWJSX73AW7NKA, seg_pos={0 0}, upload=332190]: Done uploading piece. err: <nil> Mar 18 19:34:40.432889303 (296h34m38.029660329s) [noise=true , address=79.127.219.46:4905 , piece_id=GECZVFQ26W2GHC5NFFB5X4S7KFK7LWC5NEGW2ZUAUH6MJ23ZSD3A, seg_pos={0 0}, upload=332190]: Done uploading piece. err: <nil> Mar 18 19:34:40.447990207 (296h34m38.044761231s) [noise=true , address=77.90.31.16:4753 , piece_id=6H2CQVMNGMMODIJXTXDUEEG2UQ5GIKOWJE5QLZDUC6RXXGLSTRMA, seg_pos={0 0}, upload=332190]: Done uploading piece. err: <nil> Mar 18 19:34:40.906859233 (296h34m38.503630255s) [noise=true , address=193.149.164.71:4298 , piece_id=ZTLYCQGH65XSOWC2ZDZFMXSYB6SKGCX5DS7SVWKDMO3CWB6CQDSA, seg_pos={0 0}, upload=332190]: Done uploading piece. err: <nil> Mar 18 19:34:40.913608527 (296h34m38.510379558s) [noise=true , address=193.149.164.69:4082 , piece_id=B4WBGJ4DDHDF7LSA6XNLROPT5KR76BTHBXPTYYZOMU24NUREQUTA, seg_pos={0 0}, upload=332190]: Done uploading piece. err: <nil> Mar 18 19:34:40.923706099 (296h34m38.520477123s) [noise=true , address=79.127.205.235:4282 , piece_id=UJIZ3MRUFLTSHSUUVGXHPZB66YGYT6NJLI4N32BKLDI6A6HWOBAA, seg_pos={0 0}, upload=332190]: Done uploading piece. err: <nil> Mar 18 19:34:40.926282735 (296h34m38.523053752s) [noise=true , address=193.149.164.66:4995 , piece_id=B4N4RTC3AUNDI32SOGFNCLAXOYS4JVKT7GCTRCYBHYFGHX77RL3Q, seg_pos={0 0}, upload=332190]: Done uploading piece. err: <nil> Mar 18 19:34:40.928952265 (296h34m38.525723289s) [noise=true , address=193.149.164.70:4110 , piece_id=X24XJHJHWP6LWFML5T7X2IIV7V7OHOQ2K4CZGO7BLYQAGMEJPESQ, seg_pos={0 0}, upload=332190]: Done uploading piece. err: <nil> Mar 18 19:35:40.254188258 (296h35m37.850959291s) [noise=true , address=145.79.227.45:4235 , piece_id=FGQMFHRGSP7RZZLN4QAYUTQRP4U6H5T3FCOKVBTA7UBXG4KVKUUQ, seg_pos={0 0}, upload=332190]: Done uploading piece. err: ecclient: upload failed (node:1a1sW1G1WPL5WQ2XxprWKQMsJkcb5ETSkk3ULp1TjRaZWptaT, address:145.79.227.45:4235): protocol: rpc: dial tcp 145.79.227.45:4235: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:40.254280735 (296h35m37.85105176s) [noise=true , address=145.79.227.4:4203 , piece_id=BFRKIWDBFBOXGYSEPEUHTI24YQ6KNFQ6NU5WDWWQQ44KD4CGT4LQ, seg_pos={0 0}, upload=332190]: Done uploading piece. err: ecclient: upload failed (node:18cNSsgqn3n6z5LxRcVCRzDzfFDpvptV1MKh6NdNRLW4bg9fWM, address:145.79.227.4:4203): protocol: rpc: dial tcp 145.79.227.4:4203: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:40.254371712 (296h35m37.851142741s) [noise=true , address=145.79.227.13:4835 , piece_id=SZUFWGRLFYV3YB2KTVDPPZN23PBUHQD5LI32LDYHSKUWB3URSYBQ, seg_pos={0 0}, upload=332190]: Done uploading piece. err: ecclient: upload failed (node:12WYQwhW2KDjDyweEx5FLYRkGwvW15f6eVHzbdrysiHeuKNtaoN, address:145.79.227.13:4835): protocol: rpc: dial tcp 145.79.227.13:4835: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:40.254566710 (296h35m37.851337735s) [noise=true , address=145.79.227.52:4451 , piece_id=7I6S4NZHN5GIBWSKXIGJJ2BWUSRRWO534EHLVHEWRWP6K5CWBVHA, seg_pos={0 0}, upload=332190]: Done uploading piece. err: ecclient: upload failed (node:12bC4P3xBiPFkxLvavGFVbWKnbprCStL6UJgfKSA2Wd2fX7Mbfw, address:145.79.227.52:4451): protocol: rpc: dial tcp 145.79.227.52:4451: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:40.254894056 (296h35m37.851665084s) [noise=true , address=145.79.227.26:4665 , piece_id=QLOE3K4G5XTGDTAZP4APCFVXO2QV57TU7HCZC6KB7D42D2WYXEMQ, seg_pos={0 0}, upload=332190]: Done uploading piece. err: ecclient: upload failed (node:12jP9JrCmoQSdqrzXZxkNN1ihdFkaUMjGZss6Fh19zbL3YyWj5N, address:145.79.227.26:4665): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:35:40.255167508 (296h35m37.851938533s) [noise=true , address=145.79.227.12:4791 , piece_id=2OTNUFAPZAX6VQFII6OQFT2BEAUS4PBJC36XN7FBO7OO2IF3QGNA, seg_pos={0 0}, upload=332190]: Done uploading piece. err: ecclient: upload failed (node:1MktgfjDE2jQUoUYAx1CVSjHjAkF8SZTv8a87Y7hbEs3RFCmgn, address:145.79.227.12:4791): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:35:40.255200717 (296h35m37.85197173s) [seg_pos={0 0}, upload=332190]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Mar 18 19:35:40.289900369 (296h35m37.886671392s) [upload=332190]: Done uploading segments... Mar 18 19:35:40.289938358 (296h35m37.886709382s) [upload=332190]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 1m0.038068819s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [........................................................XXXX] duration: 5.001456641s trace: https://jaeger.dev.storj.io/trace/21897b12d3827f96 FAIL (2026-03-18 19:35:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001535705s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 218.275404ms trace: https://jaeger.dev.storj.io/trace/4e61caa8af497b0f FAIL (2026-03-18 19:35:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: internal error ======LOGS====== Mar 18 18:50:37.197689801 (295h50m34.794460832s) [upload=331359]: Starting upload Mar 18 18:50:37.197768007 (295h50m34.79453904s) [upload=331359]: Done starting upload Mar 18 18:50:37.197865441 (295h50m34.79463646s) [upload=331359]: Uploading segments... Mar 18 18:50:37.198001255 (295h50m34.794772285s) [upload=331359]: Got next segment. Inline: true Mar 18 18:50:37.198009821 (295h50m34.794780854s) [upload=331359]: Waiting for error group managing segments... Mar 18 18:50:37.198092531 (295h50m34.794863559s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 18:50:37.198099572 (295h50m34.794870604s) [upload=331359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 18:50:37.198103547 (295h50m34.794874571s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 18:50:37.234689079 (295h50m34.831460108s) [upload=331359]: Done uploading segments... Mar 18 18:50:37.234743231 (295h50m34.831514254s) [upload=331359]: Upload finished. err: metaclient: internal error Duration:* 218.452783ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.025066929s trace: https://jaeger.dev.storj.io/trace/3ae12fa786938dcf FAIL (2026-03-18 19:35:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== atch item: pb.BatchRequestItem_SegmentBegin Mar 18 19:34:49.585115243 (296h34m47.181886277s) [seg_pos={0 0}, upload=332194]: Begin upload segment... Mar 18 19:34:49.585185473 (296h34m47.181956506s) [seg_pos={0 0}, upload=332194]: Done begin upload segment. Mar 18 19:34:49.585245770 (296h34m47.182016794s) [noise=true , address=145.79.227.19:4081 , piece_id=Y62AEZHKEEIA6FZQJPGZVYE6M3RUCVB53WP3WAQTDUWXSUC66VMA, seg_pos={0 0}, upload=332194]: Uploading piece... Mar 18 19:34:49.585263130 (296h34m47.182034167s) [noise=true , address=77.90.31.15:4701 , piece_id=TDUBYSKVKOCMBRN3YXMXD3H6I7GNSLTIHDQU766CPDS4AQG2X5IQ, seg_pos={0 0}, upload=332194]: Uploading piece... Mar 18 19:34:49.585334999 (296h34m47.182106033s) [noise=true , address=145.79.227.9:4370 , piece_id=FZBRB6O44WN4DVWX4ZDY6IYHHAI3RR3RHIEHET3WOMY4QA4OESOA, seg_pos={0 0}, upload=332194]: Uploading piece... Mar 18 19:34:49.585827057 (296h34m47.182598091s) [noise=true , address=145.79.227.12:4769 , piece_id=47CZYN2LUOF4BWW75XQPVEV472ZIZLO73XGWV5OP7DB36KFYZDOQ, seg_pos={0 0}, upload=332194]: Uploading piece... Mar 18 19:34:49.586059383 (296h34m47.182830421s) [noise=true , address=145.79.227.47:4270 , piece_id=MNCS6ACPDFAYVAAPINJXOECHXQT4MFFS7V7CL2MESELOUHNWZOZQ, seg_pos={0 0}, upload=332194]: Uploading piece... Mar 18 19:34:49.586311555 (296h34m47.183082581s) [noise=true , address=145.79.227.4:4190 , piece_id=S2LPGHIPDD3TX5VKB7UNJLEBD6OT6OWIBIJFPJVCYYPBD5BM2H6Q, seg_pos={0 0}, upload=332194]: Uploading piece... Mar 18 19:34:49.586689139 (296h34m47.183460165s) [noise=true , address=145.79.227.38:4979 , piece_id=BYGNORKNV43XIQT5FB5EC7BIB5LOCOPLKC4XJJVVELSZTN4M24RQ, seg_pos={0 0}, upload=332194]: Uploading piece... Mar 18 19:34:49.586702520 (296h34m47.183473546s) [noise=true , address=45.91.116.5:4300 , piece_id=O75XX45HOHA53CQ5767MWFHDGTTCWESGV6JD6B7D7UTAZAB6WEDQ, seg_pos={0 0}, upload=332194]: Uploading piece... Mar 18 19:34:49.587136004 (296h34m47.183907032s) [noise=true , address=145.79.227.33:4871 , piece_id=WA65OEMDGMHOPVT7VCFANAWWEU7T42NDM4ZU63JQYMHRB43MWAPA, seg_pos={0 0}, upload=332194]: Uploading piece... Mar 18 19:34:49.587277038 (296h34m47.184048064s) [noise=true , address=45.91.116.10:4088 , piece_id=XVKYYQ3UN32YY5FHCHJCMQ2ARNXWCVXFZHJLFIEH3UZGMV4674BQ, seg_pos={0 0}, upload=332194]: Uploading piece... Mar 18 19:34:49.587648464 (296h34m47.184419498s) [noise=true , address=145.79.227.28:4731 , piece_id=M7Z3OKKDPNJYKS56FU44BF5MVWPIDG7Z3QEVA3Y4IVNL2TK365VA, seg_pos={0 0}, upload=332194]: Uploading piece... Mar 18 19:34:49.587895124 (296h34m47.18466615s) [noise=true , address=77.90.31.18:4809 , piece_id=S2FHOT2NZDOEBDSRK5GJZ3OO3VEDBZIOXGCNLXXWWT2JEGQKH65A, seg_pos={0 0}, upload=332194]: Uploading piece... Mar 18 19:34:49.588071116 (296h34m47.18484215s) [noise=true , address=145.79.227.45:4234 , piece_id=IN6Q7PDXPNF6KDAVNMFNPVYFVP4IUZQ3F7PCBD5D45OGPDDOM7PA, seg_pos={0 0}, upload=332194]: Uploading piece... Mar 18 19:34:49.588315647 (296h34m47.185086669s) [noise=true , address=145.79.227.25:4622 , piece_id=OHFELCPQM4C536KJ7X3CQXJSA6YHIGTIWZJ57H3JZFDA6LEBJT5Q, seg_pos={0 0}, upload=332194]: Uploading piece... Mar 18 19:34:49.588736468 (296h34m47.185507498s) [noise=true , address=45.91.116.4:4940 , piece_id=5K5N2I766VTGP4G7EL2KGYCB37R3RAM2II5VNBICE4KTQX46XTXQ, seg_pos={0 0}, upload=332194]: Uploading piece... Mar 18 19:34:49.589183487 (296h34m47.185954513s) [noise=true , address=145.79.227.48:4306 , piece_id=MXVGRQQK2XW2PQJ2XIOG2K52NBIYQFKRFT3B2ZX6J3BNO6DKGU2Q, seg_pos={0 0}, upload=332194]: Uploading piece... Mar 18 19:34:49.660994267 (296h34m47.25776529s) [upload=332194]: Next returned nil segment Mar 18 19:34:49.661018759 (296h34m47.257789792s) [upload=332194]: Waiting for error group managing segments... Mar 18 19:34:49.665267026 (296h34m47.262038058s) [noise=true , address=45.91.116.10:4088 , piece_id=XVKYYQ3UN32YY5FHCHJCMQ2ARNXWCVXFZHJLFIEH3UZGMV4674BQ, seg_pos={0 0}, upload=332194]: Done uploading piece. err: <nil> Mar 18 19:34:49.668058103 (296h34m47.26482913s) [noise=true , address=77.90.31.18:4809 , piece_id=S2FHOT2NZDOEBDSRK5GJZ3OO3VEDBZIOXGCNLXXWWT2JEGQKH65A, seg_pos={0 0}, upload=332194]: Done uploading piece. err: <nil> Mar 18 19:34:49.680472485 (296h34m47.277243518s) [noise=true , address=45.91.116.5:4300 , piece_id=O75XX45HOHA53CQ5767MWFHDGTTCWESGV6JD6B7D7UTAZAB6WEDQ, seg_pos={0 0}, upload=332194]: Done uploading piece. err: <nil> Mar 18 19:34:49.680954439 (296h34m47.277725453s) [noise=true , address=45.91.116.4:4940 , piece_id=5K5N2I766VTGP4G7EL2KGYCB37R3RAM2II5VNBICE4KTQX46XTXQ, seg_pos={0 0}, upload=332194]: Done uploading piece. err: <nil> Mar 18 19:34:49.683554349 (296h34m47.280325379s) [noise=true , address=77.90.31.15:4701 , piece_id=TDUBYSKVKOCMBRN3YXMXD3H6I7GNSLTIHDQU766CPDS4AQG2X5IQ, seg_pos={0 0}, upload=332194]: Done uploading piece. err: <nil> Mar 18 19:35:19.521876928 (296h35m17.118647964s) [noise=true , address=145.79.227.28:4731 , piece_id=M7Z3OKKDPNJYKS56FU44BF5MVWPIDG7Z3QEVA3Y4IVNL2TK365VA, seg_pos={0 0}, upload=332194]: Done uploading piece. err: ecclient: upload failed (node:12NQzTwVYQa6ZzxAHQm1apkwWnqJstTnAJQyHKhBzC2jTN9kEVK, address:145.79.227.28:4731): protocol: rpc: dial tcp 145.79.227.28:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:19.521956310 (296h35m17.118727336s) [noise=true , address=145.79.227.12:4769 , piece_id=47CZYN2LUOF4BWW75XQPVEV472ZIZLO73XGWV5OP7DB36KFYZDOQ, seg_pos={0 0}, upload=332194]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:19.522029590 (296h35m17.118800616s) [noise=true , address=145.79.227.19:4081 , piece_id=Y62AEZHKEEIA6FZQJPGZVYE6M3RUCVB53WP3WAQTDUWXSUC66VMA, seg_pos={0 0}, upload=332194]: Done uploading piece. err: ecclient: upload failed (node:12t9rGfCQ7bLkNnZzaPVgkE2Y9rXsiFp1XGdeECzv9M1jk8Qqmp, address:145.79.227.19:4081): protocol: rpc: dial tcp 145.79.227.19:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:19.522075376 (296h35m17.118846402s) [noise=true , address=145.79.227.38:4979 , piece_id=BYGNORKNV43XIQT5FB5EC7BIB5LOCOPLKC4XJJVVELSZTN4M24RQ, seg_pos={0 0}, upload=332194]: Done uploading piece. err: ecclient: upload failed (node:12B518TvbbRhj6LiXxV4GwUgAoUYmWMWeTpyYerKnKuZYBjXoma, address:145.79.227.38:4979): protocol: rpc: dial tcp 145.79.227.38:4979: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:19.522114063 (296h35m17.118885095s) [noise=true , address=145.79.227.33:4871 , piece_id=WA65OEMDGMHOPVT7VCFANAWWEU7T42NDM4ZU63JQYMHRB43MWAPA, seg_pos={0 0}, upload=332194]: Done uploading piece. err: ecclient: upload failed (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, address:145.79.227.33:4871): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:35:19.522120934 (296h35m17.11889195s) [noise=true , address=145.79.227.4:4190 , piece_id=S2LPGHIPDD3TX5VKB7UNJLEBD6OT6OWIBIJFPJVCYYPBD5BM2H6Q, seg_pos={0 0}, upload=332194]: Done uploading piece. err: ecclient: upload failed (node:1eNJzs5Dc7Nv7imoQXZDp2mEVWjD8o4viGVkFvuAeZ1arynxu4, address:145.79.227.4:4190): protocol: rpc: dial tcp 145.79.227.4:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:19.522170850 (296h35m17.118941882s) [noise=true , address=145.79.227.45:4234 , piece_id=IN6Q7PDXPNF6KDAVNMFNPVYFVP4IUZQ3F7PCBD5D45OGPDDOM7PA, seg_pos={0 0}, upload=332194]: Done uploading piece. err: ecclient: upload failed (node:12uthP31WaBmn37h54Q6z9ZMtYRY6K8917Ms4kwyyADqu4EXPJy, address:145.79.227.45:4234): protocol: rpc: dial tcp 145.79.227.45:4234: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:19.522216326 (296h35m17.118987352s) [noise=true , address=145.79.227.9:4370 , piece_id=FZBRB6O44WN4DVWX4ZDY6IYHHAI3RR3RHIEHET3WOMY4QA4OESOA, seg_pos={0 0}, upload=332194]: Done uploading piece. err: ecclient: upload failed (node:12SXZN7WJvcHys9BRd4Lq8NAZ3MSgHf9ZVv7SgaiJNRosdmFVpm, address:145.79.227.9:4370): protocol: rpc: dial tcp 145.79.227.9:4370: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:19.522346000 (296h35m17.119117029s) [noise=true , address=145.79.227.47:4270 , piece_id=MNCS6ACPDFAYVAAPINJXOECHXQT4MFFS7V7CL2MESELOUHNWZOZQ, seg_pos={0 0}, upload=332194]: Done uploading piece. err: ecclient: upload failed (node:12sJfMr1nMTpizd5PZ2bZ6M8tZWhFL2ukdJ3cNSRQtdUHNJhg81, address:145.79.227.47:4270): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:35:19.522399172 (296h35m17.119170189s) [noise=true , address=145.79.227.25:4622 , piece_id=OHFELCPQM4C536KJ7X3CQXJSA6YHIGTIWZJ57H3JZFDA6LEBJT5Q, seg_pos={0 0}, upload=332194]: Done uploading piece. err: ecclient: upload failed (node:1xEK5XL3sYcip7PsQcZBQYihgnEZykdMmsKkCr2TUmbTFYncfs, address:145.79.227.25:4622): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:35:19.522422974 (296h35m17.119193995s) [noise=true , address=145.79.227.48:4306 , piece_id=MXVGRQQK2XW2PQJ2XIOG2K52NBIYQFKRFT3B2ZX6J3BNO6DKGU2Q, seg_pos={0 0}, upload=332194]: Done uploading piece. err: ecclient: upload failed (node:12amBPo7WKpjPDzi6mUWw5PcdgrcT7hsUjsSkijfrvKka1bJzNw, address:145.79.227.48:4306): protocol: rpc: dial tcp 145.79.227.48:4306: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:35:19.522502365 (296h35m17.119273383s) [seg_pos={0 0}, upload=332194]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Mar 18 19:35:19.543941750 (296h35m17.140712763s) [upload=332194]: Done uploading segments... Mar 18 19:35:19.544008198 (296h35m17.140779208s) [upload=332194]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration:* 30.025087474s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.........................................................XXX] duration: 5.002111399s trace: https://jaeger.dev.storj.io/trace/14816c12c6d66908 FAIL (2026-03-18 19:35:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.002134942s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 187.122891ms trace: https://jaeger.dev.storj.io/trace/4e61caa8af497b0f FAIL (2026-03-18 19:34:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: internal error ======LOGS====== Mar 18 18:50:37.197689801 (295h50m34.794460832s) [upload=331359]: Starting upload Mar 18 18:50:37.197768007 (295h50m34.79453904s) [upload=331359]: Done starting upload Mar 18 18:50:37.197865441 (295h50m34.79463646s) [upload=331359]: Uploading segments... Mar 18 18:50:37.198001255 (295h50m34.794772285s) [upload=331359]: Got next segment. Inline: true Mar 18 18:50:37.198009821 (295h50m34.794780854s) [upload=331359]: Waiting for error group managing segments... Mar 18 18:50:37.198092531 (295h50m34.794863559s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 18:50:37.198099572 (295h50m34.794870604s) [upload=331359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 18:50:37.198103547 (295h50m34.794874571s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 18:50:37.234689079 (295h50m34.831460108s) [upload=331359]: Done uploading segments... Mar 18 18:50:37.234743231 (295h50m34.831514254s) [upload=331359]: Upload finished. err: metaclient: internal error Duration:* 187.153546ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.001787358s trace: https://jaeger.dev.storj.io/trace/790695f985c2e6d9 FAIL (2026-03-18 19:34:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001804698s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.026363859s trace: https://jaeger.dev.storj.io/trace/70d8878ea77e6a9e FAIL (2026-03-18 19:34:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== ) [upload=332180]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 19:34:12.154200701 (296h34m9.750971728s) [upload=332180]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 19:34:12.177855949 (296h34m9.774626987s) [seg_pos={0 0}, upload=332180]: Begin upload segment... Mar 18 19:34:12.177999559 (296h34m9.774770586s) [seg_pos={0 0}, upload=332180]: Done begin upload segment. Mar 18 19:34:12.178097360 (296h34m9.774868386s) [noise=true , address=77.90.31.15:4697 , piece_id=EEMFX3OHD6NUWJ7T22KBL63NXUS7KO6TWPK5ZKTQGU2YQY7GTVHA, seg_pos={0 0}, upload=332180]: Uploading piece... Mar 18 19:34:12.179170226 (296h34m9.775941256s) [noise=true , address=145.79.227.22:4513 , piece_id=NSWNVCFLOVLCAUB4QMOEBJEBYSO3CTBO33MPBIL2Y3GZMMJSIH6A, seg_pos={0 0}, upload=332180]: Uploading piece... Mar 18 19:34:12.179200561 (296h34m9.775971595s) [noise=true , address=45.91.116.14:4224 , piece_id=J3IGYBQ7LJ2KNBAC4PJFK3AZG2PLMPDLYLLFCJCWMY6LR2KFINVQ, seg_pos={0 0}, upload=332180]: Uploading piece... Mar 18 19:34:12.179844891 (296h34m9.776615921s) [noise=true , address=145.79.227.47:4268 , piece_id=FQMHRXT3UJA6O4ZDLBIMUDTENK24I2BSOKQTVPUHI4CPGUIRJ2WQ, seg_pos={0 0}, upload=332180]: Uploading piece... Mar 18 19:34:12.179963262 (296h34m9.776734291s) [noise=true , address=77.90.31.14:4664 , piece_id=OMDD4TUV6QCDROL3NQ7UK4T7K5TLDHUQ4UL2PQ34HGH3PCMEFLTA, seg_pos={0 0}, upload=332180]: Uploading piece... Mar 18 19:34:12.180416012 (296h34m9.777187045s) [noise=true , address=145.79.227.8:4332 , piece_id=ZOCP3POQMFSWE7IK446JNHGEMTV4NM2XGYZZ7G5D5B6ZUFG3KZEA, seg_pos={0 0}, upload=332180]: Uploading piece... Mar 18 19:34:12.180517819 (296h34m9.777288853s) [noise=true , address=145.79.227.50:4374 , piece_id=U4IJUN64YDONYXXZADRBHG7HUIF762LTJHUCE3OHLPSB2M6UEL6Q, seg_pos={0 0}, upload=332180]: Uploading piece... Mar 18 19:34:12.180946718 (296h34m9.777717747s) [noise=true , address=145.79.227.15:4875 , piece_id=JDK5DPH5WXVYGPSYAZJXUDZIJ56S4UUFV7DGRTKPB6P472NZA5SQ, seg_pos={0 0}, upload=332180]: Uploading piece... Mar 18 19:34:12.181050270 (296h34m9.777821301s) [noise=true , address=145.79.227.18:4008 , piece_id=3WP5LSR5O4QBY2GI5E5OWTNSUCO5DORDQIV3OAM4BFMZ5DBJKCIA, seg_pos={0 0}, upload=332180]: Uploading piece... Mar 18 19:34:12.181424145 (296h34m9.778195168s) [noise=true , address=45.91.116.13:4154 , piece_id=E52TI6JU5KIHUGFOVNLBNKXXFYCGXQYRUR7UPBNKNLMVUFLBXS6A, seg_pos={0 0}, upload=332180]: Uploading piece... Mar 18 19:34:12.181468437 (296h34m9.77823947s) [noise=true , address=145.79.227.4:4191 , piece_id=KOL7JEHKZQCK2DUQMJKDNSLIHVQZCI56RBWLTZ4JOOMMW36TM3ZQ, seg_pos={0 0}, upload=332180]: Uploading piece... Mar 18 19:34:12.181849112 (296h34m9.778620133s) [noise=true , address=145.79.227.19:4082 , piece_id=LIO5HVQRTAZQR4TJYOKW2LDL5QX3UX6HBKRMQX4IJ3L5666HSSKA, seg_pos={0 0}, upload=332180]: Uploading piece... Mar 18 19:34:12.181983115 (296h34m9.778754149s) [noise=true , address=145.79.227.49:4340 , piece_id=QKYI2B6L7ATUAGQHS2MVKGZN5N57QQJ5EXWSJYGKFRPIOBLHYVQQ, seg_pos={0 0}, upload=332180]: Uploading piece... Mar 18 19:34:12.182439604 (296h34m9.77921064s) [noise=true , address=45.91.116.8:4070 , piece_id=44ZCUP2SNDQZCHFN54GWQQPGSLQCBQASQPACCQXZSV3GRLR5SOIA, seg_pos={0 0}, upload=332180]: Uploading piece... Mar 18 19:34:12.183810456 (296h34m9.780581553s) [noise=true , address=77.90.31.17:4770 , piece_id=LNQKUCJM3BEUPCV7OW3YROW6Y3I7DX7XTUA3GY4Q6RKPWI2QBXOA, seg_pos={0 0}, upload=332180]: Uploading piece... Mar 18 19:34:12.184293474 (296h34m9.78106449s) [noise=true , address=145.79.227.21:4473 , piece_id=UTVSJ5MAT5AYNHZNVYWG4SAG4AEJ3WAMI3CCRG3IOFGZ2POYAMQQ, seg_pos={0 0}, upload=332180]: Uploading piece... Mar 18 19:34:12.260495327 (296h34m9.857266358s) [upload=332180]: Next returned nil segment Mar 18 19:34:12.260510623 (296h34m9.857281651s) [upload=332180]: Waiting for error group managing segments... Mar 18 19:34:12.266981006 (296h34m9.86375203s) [noise=true , address=45.91.116.14:4224 , piece_id=J3IGYBQ7LJ2KNBAC4PJFK3AZG2PLMPDLYLLFCJCWMY6LR2KFINVQ, seg_pos={0 0}, upload=332180]: Done uploading piece. err: <nil> Mar 18 19:34:12.267295745 (296h34m9.864066768s) [noise=true , address=45.91.116.8:4070 , piece_id=44ZCUP2SNDQZCHFN54GWQQPGSLQCBQASQPACCQXZSV3GRLR5SOIA, seg_pos={0 0}, upload=332180]: Done uploading piece. err: <nil> Mar 18 19:34:12.270302472 (296h34m9.867073495s) [noise=true , address=77.90.31.15:4697 , piece_id=EEMFX3OHD6NUWJ7T22KBL63NXUS7KO6TWPK5ZKTQGU2YQY7GTVHA, seg_pos={0 0}, upload=332180]: Done uploading piece. err: <nil> Mar 18 19:34:12.270365583 (296h34m9.86713661s) [noise=true , address=77.90.31.14:4664 , piece_id=OMDD4TUV6QCDROL3NQ7UK4T7K5TLDHUQ4UL2PQ34HGH3PCMEFLTA, seg_pos={0 0}, upload=332180]: Done uploading piece. err: <nil> Mar 18 19:34:12.287050587 (296h34m9.88382161s) [noise=true , address=45.91.116.13:4154 , piece_id=E52TI6JU5KIHUGFOVNLBNKXXFYCGXQYRUR7UPBNKNLMVUFLBXS6A, seg_pos={0 0}, upload=332180]: Done uploading piece. err: <nil> Mar 18 19:34:12.293647634 (296h34m9.890418719s) [noise=true , address=77.90.31.17:4770 , piece_id=LNQKUCJM3BEUPCV7OW3YROW6Y3I7DX7XTUA3GY4Q6RKPWI2QBXOA, seg_pos={0 0}, upload=332180]: Done uploading piece. err: <nil> Mar 18 19:34:42.113612186 (296h34m39.710383218s) [noise=true , address=145.79.227.18:4008 , piece_id=3WP5LSR5O4QBY2GI5E5OWTNSUCO5DORDQIV3OAM4BFMZ5DBJKCIA, seg_pos={0 0}, upload=332180]: Done uploading piece. err: ecclient: upload failed (node:148CqQmrsJSfoGByHYRPoZmvP1VgQQpaMVLb7RgRQvNBuo7air, address:145.79.227.18:4008): protocol: rpc: dial tcp 145.79.227.18:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:34:42.113661220 (296h34m39.71043225s) [noise=true , address=145.79.227.8:4332 , piece_id=ZOCP3POQMFSWE7IK446JNHGEMTV4NM2XGYZZ7G5D5B6ZUFG3KZEA, seg_pos={0 0}, upload=332180]: Done uploading piece. err: ecclient: upload failed (node:1khNvxTmr2av1PFmxCv4cEcMv25F9y41bndYPQKuP2LCx7WoMQ, address:145.79.227.8:4332): protocol: rpc: dial tcp 145.79.227.8:4332: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:34:42.113865839 (296h34m39.710636871s) [noise=true , address=145.79.227.15:4875 , piece_id=JDK5DPH5WXVYGPSYAZJXUDZIJ56S4UUFV7DGRTKPB6P472NZA5SQ, seg_pos={0 0}, upload=332180]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:34:42.114331282 (296h34m39.711102314s) [noise=true , address=145.79.227.22:4513 , piece_id=NSWNVCFLOVLCAUB4QMOEBJEBYSO3CTBO33MPBIL2Y3GZMMJSIH6A, seg_pos={0 0}, upload=332180]: Done uploading piece. err: ecclient: upload failed (node:12BykEXT7spgn4jkPucFmPvjvgKPKi5fZgRAeMTbfecZ7jCNori, address:145.79.227.22:4513): protocol: rpc: dial tcp 145.79.227.22:4513: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:34:42.114645213 (296h34m39.711416256s) [noise=true , address=145.79.227.19:4082 , piece_id=LIO5HVQRTAZQR4TJYOKW2LDL5QX3UX6HBKRMQX4IJ3L5666HSSKA, seg_pos={0 0}, upload=332180]: Done uploading piece. err: ecclient: upload failed (node:12DKsNCtqxLvpJwLszdLs74V3eb8NH2VAV9V3bngzUczawmNoLK, address:145.79.227.19:4082): protocol: rpc: dial tcp 145.79.227.19:4082: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:34:42.114928201 (296h34m39.711699237s) [noise=true , address=145.79.227.49:4340 , piece_id=QKYI2B6L7ATUAGQHS2MVKGZN5N57QQJ5EXWSJYGKFRPIOBLHYVQQ, seg_pos={0 0}, upload=332180]: Done uploading piece. err: ecclient: upload failed (node:1XeA4A8wbmv7wgPjRQnNbP9xgDQz4MzJH7PasaYsTrsMzTJRhC, address:145.79.227.49:4340): protocol: rpc: dial tcp 145.79.227.49:4340: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:34:42.115080970 (296h34m39.711852005s) [noise=true , address=145.79.227.50:4374 , piece_id=U4IJUN64YDONYXXZADRBHG7HUIF762LTJHUCE3OHLPSB2M6UEL6Q, seg_pos={0 0}, upload=332180]: Done uploading piece. err: ecclient: upload failed (node:12QRixhWaUwXPdjcWKYHbiknME5JVomWYABY6egDD6VKwaTPB8K, address:145.79.227.50:4374): protocol: rpc: dial tcp 145.79.227.50:4374: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:34:42.115226335 (296h34m39.711997372s) [noise=true , address=145.79.227.21:4473 , piece_id=UTVSJ5MAT5AYNHZNVYWG4SAG4AEJ3WAMI3CCRG3IOFGZ2POYAMQQ, seg_pos={0 0}, upload=332180]: Done uploading piece. err: ecclient: upload failed (node:12DDe4jT3YeUH8qMQgapcvzMnFs8UFB8fTmubho4ZQf5tYVoxuk, address:145.79.227.21:4473): protocol: rpc: dial tcp 145.79.227.21:4473: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:34:42.115285516 (296h34m39.712056528s) [noise=true , address=145.79.227.47:4268 , piece_id=FQMHRXT3UJA6O4ZDLBIMUDTENK24I2BSOKQTVPUHI4CPGUIRJ2WQ, seg_pos={0 0}, upload=332180]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: rpc: dial tcp 145.79.227.47:4268: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:34:42.115568363 (296h34m39.712339389s) [noise=true , address=145.79.227.4:4191 , piece_id=KOL7JEHKZQCK2DUQMJKDNSLIHVQZCI56RBWLTZ4JOOMMW36TM3ZQ, seg_pos={0 0}, upload=332180]: Done uploading piece. err: ecclient: upload failed (node:1sVWXyKPnLTobAhk6GXZTTP67FbyY3XJ1e2svTiUEa3rrghd2q, address:145.79.227.4:4191): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:34:42.115623783 (296h34m39.712394799s) [seg_pos={0 0}, upload=332180]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Mar 18 19:34:42.137129976 (296h34m39.733900997s) [upload=332180]: Done uploading segments... Mar 18 19:34:42.137202000 (296h34m39.733973024s) [upload=332180]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 30.026389112s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 197.25743ms trace: https://jaeger.dev.storj.io/trace/4e61caa8af497b0f FAIL (2026-03-18 19:34:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: internal error ======LOGS====== Mar 18 18:50:37.197689801 (295h50m34.794460832s) [upload=331359]: Starting upload Mar 18 18:50:37.197768007 (295h50m34.79453904s) [upload=331359]: Done starting upload Mar 18 18:50:37.197865441 (295h50m34.79463646s) [upload=331359]: Uploading segments... Mar 18 18:50:37.198001255 (295h50m34.794772285s) [upload=331359]: Got next segment. Inline: true Mar 18 18:50:37.198009821 (295h50m34.794780854s) [upload=331359]: Waiting for error group managing segments... Mar 18 18:50:37.198092531 (295h50m34.794863559s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 18:50:37.198099572 (295h50m34.794870604s) [upload=331359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 18:50:37.198103547 (295h50m34.794874571s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 18:50:37.234689079 (295h50m34.831460108s) [upload=331359]: Done uploading segments... Mar 18 18:50:37.234743231 (295h50m34.831514254s) [upload=331359]: Upload finished. err: metaclient: internal error Duration:* 197.347143ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..........................................................XX] duration: 5.001335884s trace: https://jaeger.dev.storj.io/trace/7c7789b338903c51 FAIL (2026-03-18 19:34:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001362838s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................................................X] duration: 5.005316824s trace: https://jaeger.dev.storj.io/trace/2b0c7911c742ce35 FAIL (2026-03-18 19:33:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.005342513s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 213.031923ms trace: https://jaeger.dev.storj.io/trace/4e61caa8af497b0f FAIL (2026-03-18 19:33:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: internal error ======LOGS====== Mar 18 18:50:37.197689801 (295h50m34.794460832s) [upload=331359]: Starting upload Mar 18 18:50:37.197768007 (295h50m34.79453904s) [upload=331359]: Done starting upload Mar 18 18:50:37.197865441 (295h50m34.79463646s) [upload=331359]: Uploading segments... Mar 18 18:50:37.198001255 (295h50m34.794772285s) [upload=331359]: Got next segment. Inline: true Mar 18 18:50:37.198009821 (295h50m34.794780854s) [upload=331359]: Waiting for error group managing segments... Mar 18 18:50:37.198092531 (295h50m34.794863559s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 18:50:37.198099572 (295h50m34.794870604s) [upload=331359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 18:50:37.198103547 (295h50m34.794874571s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 18:50:37.234689079 (295h50m34.831460108s) [upload=331359]: Done uploading segments... Mar 18 18:50:37.234743231 (295h50m34.831514254s) [upload=331359]: Upload finished. err: metaclient: internal error Duration:* 213.063128ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 207.907431ms trace: https://jaeger.dev.storj.io/trace/4e61caa8af497b0f FAIL (2026-03-18 19:33:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: internal error ======LOGS====== Mar 18 18:50:37.197689801 (295h50m34.794460832s) [upload=331359]: Starting upload Mar 18 18:50:37.197768007 (295h50m34.79453904s) [upload=331359]: Done starting upload Mar 18 18:50:37.197865441 (295h50m34.79463646s) [upload=331359]: Uploading segments... Mar 18 18:50:37.198001255 (295h50m34.794772285s) [upload=331359]: Got next segment. Inline: true Mar 18 18:50:37.198009821 (295h50m34.794780854s) [upload=331359]: Waiting for error group managing segments... Mar 18 18:50:37.198092531 (295h50m34.794863559s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 18:50:37.198099572 (295h50m34.794870604s) [upload=331359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 18:50:37.198103547 (295h50m34.794874571s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 18:50:37.234689079 (295h50m34.831460108s) [upload=331359]: Done uploading segments... Mar 18 18:50:37.234743231 (295h50m34.831514254s) [upload=331359]: Upload finished. err: metaclient: internal error Duration:* 207.93773ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 220.93778ms trace: https://jaeger.dev.storj.io/trace/4e61caa8af497b0f FAIL (2026-03-18 19:32:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: internal error ======LOGS====== Mar 18 18:50:37.197689801 (295h50m34.794460832s) [upload=331359]: Starting upload Mar 18 18:50:37.197768007 (295h50m34.79453904s) [upload=331359]: Done starting upload Mar 18 18:50:37.197865441 (295h50m34.79463646s) [upload=331359]: Uploading segments... Mar 18 18:50:37.198001255 (295h50m34.794772285s) [upload=331359]: Got next segment. Inline: true Mar 18 18:50:37.198009821 (295h50m34.794780854s) [upload=331359]: Waiting for error group managing segments... Mar 18 18:50:37.198092531 (295h50m34.794863559s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 18:50:37.198099572 (295h50m34.794870604s) [upload=331359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 18:50:37.198103547 (295h50m34.794874571s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 18:50:37.234689079 (295h50m34.831460108s) [upload=331359]: Done uploading segments... Mar 18 18:50:37.234743231 (295h50m34.831514254s) [upload=331359]: Upload finished. err: metaclient: internal error Duration:* 220.980368ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 413.175097ms trace: https://jaeger.dev.storj.io/trace/4e61caa8af497b0f FAIL (2026-03-18 19:31:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: internal error ======LOGS====== Mar 18 18:50:37.197689801 (295h50m34.794460832s) [upload=331359]: Starting upload Mar 18 18:50:37.197768007 (295h50m34.79453904s) [upload=331359]: Done starting upload Mar 18 18:50:37.197865441 (295h50m34.79463646s) [upload=331359]: Uploading segments... Mar 18 18:50:37.198001255 (295h50m34.794772285s) [upload=331359]: Got next segment. Inline: true Mar 18 18:50:37.198009821 (295h50m34.794780854s) [upload=331359]: Waiting for error group managing segments... Mar 18 18:50:37.198092531 (295h50m34.794863559s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 18:50:37.198099572 (295h50m34.794870604s) [upload=331359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 18:50:37.198103547 (295h50m34.794874571s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 18:50:37.234689079 (295h50m34.831460108s) [upload=331359]: Done uploading segments... Mar 18 18:50:37.234743231 (295h50m34.831514254s) [upload=331359]: Upload finished. err: metaclient: internal error Duration:* 413.207611ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 199.285644ms trace: https://jaeger.dev.storj.io/trace/4e61caa8af497b0f FAIL (2026-03-18 19:30:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: internal error ======LOGS====== Mar 18 18:50:37.197689801 (295h50m34.794460832s) [upload=331359]: Starting upload Mar 18 18:50:37.197768007 (295h50m34.79453904s) [upload=331359]: Done starting upload Mar 18 18:50:37.197865441 (295h50m34.79463646s) [upload=331359]: Uploading segments... Mar 18 18:50:37.198001255 (295h50m34.794772285s) [upload=331359]: Got next segment. Inline: true Mar 18 18:50:37.198009821 (295h50m34.794780854s) [upload=331359]: Waiting for error group managing segments... Mar 18 18:50:37.198092531 (295h50m34.794863559s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 18:50:37.198099572 (295h50m34.794870604s) [upload=331359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 18:50:37.198103547 (295h50m34.794874571s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 18:50:37.234689079 (295h50m34.831460108s) [upload=331359]: Done uploading segments... Mar 18 18:50:37.234743231 (295h50m34.831514254s) [upload=331359]: Upload finished. err: metaclient: internal error Duration:* 199.316685ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 227.899807ms trace: https://jaeger.dev.storj.io/trace/4e61caa8af497b0f FAIL (2026-03-18 19:29:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: internal error ======LOGS====== Mar 18 18:50:37.197689801 (295h50m34.794460832s) [upload=331359]: Starting upload Mar 18 18:50:37.197768007 (295h50m34.79453904s) [upload=331359]: Done starting upload Mar 18 18:50:37.197865441 (295h50m34.79463646s) [upload=331359]: Uploading segments... Mar 18 18:50:37.198001255 (295h50m34.794772285s) [upload=331359]: Got next segment. Inline: true Mar 18 18:50:37.198009821 (295h50m34.794780854s) [upload=331359]: Waiting for error group managing segments... Mar 18 18:50:37.198092531 (295h50m34.794863559s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 18:50:37.198099572 (295h50m34.794870604s) [upload=331359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 18:50:37.198103547 (295h50m34.794874571s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 18:50:37.234689079 (295h50m34.831460108s) [upload=331359]: Done uploading segments... Mar 18 18:50:37.234743231 (295h50m34.831514254s) [upload=331359]: Upload finished. err: metaclient: internal error Duration:* 227.928003ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 212.615044ms trace: https://jaeger.dev.storj.io/trace/4e61caa8af497b0f FAIL (2026-03-18 19:28:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: internal error ======LOGS====== Mar 18 18:50:37.197689801 (295h50m34.794460832s) [upload=331359]: Starting upload Mar 18 18:50:37.197768007 (295h50m34.79453904s) [upload=331359]: Done starting upload Mar 18 18:50:37.197865441 (295h50m34.79463646s) [upload=331359]: Uploading segments... Mar 18 18:50:37.198001255 (295h50m34.794772285s) [upload=331359]: Got next segment. Inline: true Mar 18 18:50:37.198009821 (295h50m34.794780854s) [upload=331359]: Waiting for error group managing segments... Mar 18 18:50:37.198092531 (295h50m34.794863559s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 18:50:37.198099572 (295h50m34.794870604s) [upload=331359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 18:50:37.198103547 (295h50m34.794874571s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 18:50:37.234689079 (295h50m34.831460108s) [upload=331359]: Done uploading segments... Mar 18 18:50:37.234743231 (295h50m34.831514254s) [upload=331359]: Upload finished. err: metaclient: internal error Duration:* 212.644632ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 268.640765ms trace: https://jaeger.dev.storj.io/trace/4e61caa8af497b0f FAIL (2026-03-18 19:27:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: internal error ======LOGS====== Mar 18 18:50:37.197689801 (295h50m34.794460832s) [upload=331359]: Starting upload Mar 18 18:50:37.197768007 (295h50m34.79453904s) [upload=331359]: Done starting upload Mar 18 18:50:37.197865441 (295h50m34.79463646s) [upload=331359]: Uploading segments... Mar 18 18:50:37.198001255 (295h50m34.794772285s) [upload=331359]: Got next segment. Inline: true Mar 18 18:50:37.198009821 (295h50m34.794780854s) [upload=331359]: Waiting for error group managing segments... Mar 18 18:50:37.198092531 (295h50m34.794863559s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 18:50:37.198099572 (295h50m34.794870604s) [upload=331359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 18:50:37.198103547 (295h50m34.794874571s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 18:50:37.234689079 (295h50m34.831460108s) [upload=331359]: Done uploading segments... Mar 18 18:50:37.234743231 (295h50m34.831514254s) [upload=331359]: Upload finished. err: metaclient: internal error Duration:* 268.691011ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 263.842547ms trace: https://jaeger.dev.storj.io/trace/4e61caa8af497b0f FAIL (2026-03-18 19:26:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: internal error ======LOGS====== Mar 18 18:50:37.197689801 (295h50m34.794460832s) [upload=331359]: Starting upload Mar 18 18:50:37.197768007 (295h50m34.79453904s) [upload=331359]: Done starting upload Mar 18 18:50:37.197865441 (295h50m34.79463646s) [upload=331359]: Uploading segments... Mar 18 18:50:37.198001255 (295h50m34.794772285s) [upload=331359]: Got next segment. Inline: true Mar 18 18:50:37.198009821 (295h50m34.794780854s) [upload=331359]: Waiting for error group managing segments... Mar 18 18:50:37.198092531 (295h50m34.794863559s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 18:50:37.198099572 (295h50m34.794870604s) [upload=331359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 18:50:37.198103547 (295h50m34.794874571s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 18:50:37.234689079 (295h50m34.831460108s) [upload=331359]: Done uploading segments... Mar 18 18:50:37.234743231 (295h50m34.831514254s) [upload=331359]: Upload finished. err: metaclient: internal error Duration:* 263.87732ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 229.399471ms trace: https://jaeger.dev.storj.io/trace/4e61caa8af497b0f FAIL (2026-03-18 19:25:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: internal error ======LOGS====== Mar 18 18:50:37.197689801 (295h50m34.794460832s) [upload=331359]: Starting upload Mar 18 18:50:37.197768007 (295h50m34.79453904s) [upload=331359]: Done starting upload Mar 18 18:50:37.197865441 (295h50m34.79463646s) [upload=331359]: Uploading segments... Mar 18 18:50:37.198001255 (295h50m34.794772285s) [upload=331359]: Got next segment. Inline: true Mar 18 18:50:37.198009821 (295h50m34.794780854s) [upload=331359]: Waiting for error group managing segments... Mar 18 18:50:37.198092531 (295h50m34.794863559s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 18:50:37.198099572 (295h50m34.794870604s) [upload=331359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 18:50:37.198103547 (295h50m34.794874571s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 18:50:37.234689079 (295h50m34.831460108s) [upload=331359]: Done uploading segments... Mar 18 18:50:37.234743231 (295h50m34.831514254s) [upload=331359]: Upload finished. err: metaclient: internal error Duration:* 229.448272ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 204.646648ms trace: https://jaeger.dev.storj.io/trace/4e61caa8af497b0f FAIL (2026-03-18 19:24:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: internal error ======LOGS====== Mar 18 18:50:37.197689801 (295h50m34.794460832s) [upload=331359]: Starting upload Mar 18 18:50:37.197768007 (295h50m34.79453904s) [upload=331359]: Done starting upload Mar 18 18:50:37.197865441 (295h50m34.79463646s) [upload=331359]: Uploading segments... Mar 18 18:50:37.198001255 (295h50m34.794772285s) [upload=331359]: Got next segment. Inline: true Mar 18 18:50:37.198009821 (295h50m34.794780854s) [upload=331359]: Waiting for error group managing segments... Mar 18 18:50:37.198092531 (295h50m34.794863559s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 18:50:37.198099572 (295h50m34.794870604s) [upload=331359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 18:50:37.198103547 (295h50m34.794874571s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 18:50:37.234689079 (295h50m34.831460108s) [upload=331359]: Done uploading segments... Mar 18 18:50:37.234743231 (295h50m34.831514254s) [upload=331359]: Upload finished. err: metaclient: internal error Duration:* 204.740186ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 250.970365ms trace: https://jaeger.dev.storj.io/trace/4e61caa8af497b0f FAIL (2026-03-18 19:24:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: internal error ======LOGS====== Mar 18 18:50:37.197689801 (295h50m34.794460832s) [upload=331359]: Starting upload Mar 18 18:50:37.197768007 (295h50m34.79453904s) [upload=331359]: Done starting upload Mar 18 18:50:37.197865441 (295h50m34.79463646s) [upload=331359]: Uploading segments... Mar 18 18:50:37.198001255 (295h50m34.794772285s) [upload=331359]: Got next segment. Inline: true Mar 18 18:50:37.198009821 (295h50m34.794780854s) [upload=331359]: Waiting for error group managing segments... Mar 18 18:50:37.198092531 (295h50m34.794863559s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 18:50:37.198099572 (295h50m34.794870604s) [upload=331359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 18:50:37.198103547 (295h50m34.794874571s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 18:50:37.234689079 (295h50m34.831460108s) [upload=331359]: Done uploading segments... Mar 18 18:50:37.234743231 (295h50m34.831514254s) [upload=331359]: Upload finished. err: metaclient: internal error Duration:* 251.01016ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 244.768442ms trace: https://jaeger.dev.storj.io/trace/4e61caa8af497b0f FAIL (2026-03-18 19:23:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: internal error ======LOGS====== Mar 18 18:50:37.197689801 (295h50m34.794460832s) [upload=331359]: Starting upload Mar 18 18:50:37.197768007 (295h50m34.79453904s) [upload=331359]: Done starting upload Mar 18 18:50:37.197865441 (295h50m34.79463646s) [upload=331359]: Uploading segments... Mar 18 18:50:37.198001255 (295h50m34.794772285s) [upload=331359]: Got next segment. Inline: true Mar 18 18:50:37.198009821 (295h50m34.794780854s) [upload=331359]: Waiting for error group managing segments... Mar 18 18:50:37.198092531 (295h50m34.794863559s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 18:50:37.198099572 (295h50m34.794870604s) [upload=331359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 18:50:37.198103547 (295h50m34.794874571s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 18:50:37.234689079 (295h50m34.831460108s) [upload=331359]: Done uploading segments... Mar 18 18:50:37.234743231 (295h50m34.831514254s) [upload=331359]: Upload finished. err: metaclient: internal error Duration:* 244.819685ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 222.155142ms trace: https://jaeger.dev.storj.io/trace/4e61caa8af497b0f FAIL (2026-03-18 19:22:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: internal error ======LOGS====== Mar 18 18:50:37.197689801 (295h50m34.794460832s) [upload=331359]: Starting upload Mar 18 18:50:37.197768007 (295h50m34.79453904s) [upload=331359]: Done starting upload Mar 18 18:50:37.197865441 (295h50m34.79463646s) [upload=331359]: Uploading segments... Mar 18 18:50:37.198001255 (295h50m34.794772285s) [upload=331359]: Got next segment. Inline: true Mar 18 18:50:37.198009821 (295h50m34.794780854s) [upload=331359]: Waiting for error group managing segments... Mar 18 18:50:37.198092531 (295h50m34.794863559s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 18:50:37.198099572 (295h50m34.794870604s) [upload=331359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 18:50:37.198103547 (295h50m34.794874571s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 18:50:37.234689079 (295h50m34.831460108s) [upload=331359]: Done uploading segments... Mar 18 18:50:37.234743231 (295h50m34.831514254s) [upload=331359]: Upload finished. err: metaclient: internal error Duration:* 222.186108ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 204.332374ms trace: https://jaeger.dev.storj.io/trace/4e61caa8af497b0f FAIL (2026-03-18 19:21:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: internal error ======LOGS====== Mar 18 18:50:37.197689801 (295h50m34.794460832s) [upload=331359]: Starting upload Mar 18 18:50:37.197768007 (295h50m34.79453904s) [upload=331359]: Done starting upload Mar 18 18:50:37.197865441 (295h50m34.79463646s) [upload=331359]: Uploading segments... Mar 18 18:50:37.198001255 (295h50m34.794772285s) [upload=331359]: Got next segment. Inline: true Mar 18 18:50:37.198009821 (295h50m34.794780854s) [upload=331359]: Waiting for error group managing segments... Mar 18 18:50:37.198092531 (295h50m34.794863559s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 18:50:37.198099572 (295h50m34.794870604s) [upload=331359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 18:50:37.198103547 (295h50m34.794874571s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 18:50:37.234689079 (295h50m34.831460108s) [upload=331359]: Done uploading segments... Mar 18 18:50:37.234743231 (295h50m34.831514254s) [upload=331359]: Upload finished. err: metaclient: internal error Duration:* 204.35933ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 220.97796ms trace: https://jaeger.dev.storj.io/trace/4e61caa8af497b0f FAIL (2026-03-18 19:20:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: internal error ======LOGS====== Mar 18 18:50:37.197689801 (295h50m34.794460832s) [upload=331359]: Starting upload Mar 18 18:50:37.197768007 (295h50m34.79453904s) [upload=331359]: Done starting upload Mar 18 18:50:37.197865441 (295h50m34.79463646s) [upload=331359]: Uploading segments... Mar 18 18:50:37.198001255 (295h50m34.794772285s) [upload=331359]: Got next segment. Inline: true Mar 18 18:50:37.198009821 (295h50m34.794780854s) [upload=331359]: Waiting for error group managing segments... Mar 18 18:50:37.198092531 (295h50m34.794863559s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 18:50:37.198099572 (295h50m34.794870604s) [upload=331359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 18:50:37.198103547 (295h50m34.794874571s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 18:50:37.234689079 (295h50m34.831460108s) [upload=331359]: Done uploading segments... Mar 18 18:50:37.234743231 (295h50m34.831514254s) [upload=331359]: Upload finished. err: metaclient: internal error Duration:* 221.020001ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 196.335358ms trace: https://jaeger.dev.storj.io/trace/4e61caa8af497b0f FAIL (2026-03-18 19:19:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: internal error ======LOGS====== Mar 18 18:50:37.197689801 (295h50m34.794460832s) [upload=331359]: Starting upload Mar 18 18:50:37.197768007 (295h50m34.79453904s) [upload=331359]: Done starting upload Mar 18 18:50:37.197865441 (295h50m34.79463646s) [upload=331359]: Uploading segments... Mar 18 18:50:37.198001255 (295h50m34.794772285s) [upload=331359]: Got next segment. Inline: true Mar 18 18:50:37.198009821 (295h50m34.794780854s) [upload=331359]: Waiting for error group managing segments... Mar 18 18:50:37.198092531 (295h50m34.794863559s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 18:50:37.198099572 (295h50m34.794870604s) [upload=331359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 18:50:37.198103547 (295h50m34.794874571s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 18:50:37.234689079 (295h50m34.831460108s) [upload=331359]: Done uploading segments... Mar 18 18:50:37.234743231 (295h50m34.831514254s) [upload=331359]: Upload finished. err: metaclient: internal error Duration:* 196.365144ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 219.231062ms trace: https://jaeger.dev.storj.io/trace/4e61caa8af497b0f FAIL (2026-03-18 19:19:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: internal error ======LOGS====== Mar 18 18:50:37.197689801 (295h50m34.794460832s) [upload=331359]: Starting upload Mar 18 18:50:37.197768007 (295h50m34.79453904s) [upload=331359]: Done starting upload Mar 18 18:50:37.197865441 (295h50m34.79463646s) [upload=331359]: Uploading segments... Mar 18 18:50:37.198001255 (295h50m34.794772285s) [upload=331359]: Got next segment. Inline: true Mar 18 18:50:37.198009821 (295h50m34.794780854s) [upload=331359]: Waiting for error group managing segments... Mar 18 18:50:37.198092531 (295h50m34.794863559s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 18:50:37.198099572 (295h50m34.794870604s) [upload=331359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 18:50:37.198103547 (295h50m34.794874571s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 18:50:37.234689079 (295h50m34.831460108s) [upload=331359]: Done uploading segments... Mar 18 18:50:37.234743231 (295h50m34.831514254s) [upload=331359]: Upload finished. err: metaclient: internal error Duration:* 219.263763ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 218.64384ms trace: https://jaeger.dev.storj.io/trace/4e61caa8af497b0f FAIL (2026-03-18 19:18:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: internal error ======LOGS====== Mar 18 18:50:37.197689801 (295h50m34.794460832s) [upload=331359]: Starting upload Mar 18 18:50:37.197768007 (295h50m34.79453904s) [upload=331359]: Done starting upload Mar 18 18:50:37.197865441 (295h50m34.79463646s) [upload=331359]: Uploading segments... Mar 18 18:50:37.198001255 (295h50m34.794772285s) [upload=331359]: Got next segment. Inline: true Mar 18 18:50:37.198009821 (295h50m34.794780854s) [upload=331359]: Waiting for error group managing segments... Mar 18 18:50:37.198092531 (295h50m34.794863559s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 18:50:37.198099572 (295h50m34.794870604s) [upload=331359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 18:50:37.198103547 (295h50m34.794874571s) [upload=331359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 18:50:37.234689079 (295h50m34.831460108s) [upload=331359]: Done uploading segments... Mar 18 18:50:37.234743231 (295h50m34.831514254s) [upload=331359]: Upload finished. err: metaclient: internal error Duration:* 218.677964ms core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)