Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 505.886777ms FAIL (2025-07-25 10:15:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 505.91249ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......XX....................................................] duration: 1.702990561s FAIL (2025-07-25 10:15:21 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 1.703023909s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 780.771438ms FAIL (2025-07-25 10:15:21 UTC) orbiter warning: ======ERROR====== uplink: permission denied (bucket: metaclient: Unauthorized API credentials) ======LOGS====== h item: pb.BatchRequestItem_SegmentBegin Jul 25 09:35:15.918314780 (21h59m58.763116293s) [seg_pos={0 0}, upload=21044]: Begin upload segment... Jul 25 09:35:15.918358070 (21h59m58.763159586s) [seg_pos={0 0}, upload=21044]: Done begin upload segment. Jul 25 09:35:15.918382251 (21h59m58.763183776s) [noise=true , address=77.90.31.14:4678 , piece_id=HIZS7J3DY6Q33PO2AI6UNXQ5S54L6DGG6ZAP3YKZZSEUINZCXEPQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.918407152 (21h59m58.763208685s) [noise=true , address=45.91.116.6:4994 , piece_id=JPWTQR5RKBZACY6XJDNQAEUCFDNYFLC7BYZCLHOZCCFJPXEKOSMA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.918842345 (21h59m58.763643878s) [noise=true , address=77.90.31.16:4751 , piece_id=DCODR5LGROHG7DVIUSRLFRRBKR672JGYQ3JOOTJSLL2NVLY6L62Q, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.918965591 (21h59m58.763767123s) [noise=true , address=77.90.31.15:4720 , piece_id=FL7PFV4B2TZ54WTT3YYU3NPGT7RBNGGHRUS4QTOARDZASB7PKAGQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.919180183 (21h59m58.763981708s) [noise=true , address=77.90.31.18:4830 , piece_id=2Z6ELONZADYWJJZPQUWKPTARWQI5D4CRE7XDIQBCUVVWAOUOYFGA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.919362724 (21h59m58.76416425s) [noise=true , address=79.127.205.235:4690 , piece_id=4MLADNII3WTUAMY5KKWUM2XSVFDR3NSLA32EMVJ3AIDOJENSF3SA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.919457169 (21h59m58.764258696s) [noise=true , address=79.127.205.246:4263 , piece_id=2RB4RONQXXPKZIETGCVEOM2JQUC5EIKETQA6VKCSSHF6ARLG2BRQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.919677786 (21h59m58.764479318s) [noise=true , address=104.36.82.15:4284 , piece_id=3OMO32MXTLOB6JUNQQVR27UNL5CHYN5XWWG3TO3DUOOIMEWYYY3A, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.919849288 (21h59m58.764650816s) [noise=true , address=45.91.116.10:4099 , piece_id=CWLXVFK56DYPPB3ED5INVEHKBUPW6MSKZLAA7SAFQU3YFT3UNHXQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.920171246 (21h59m58.764972769s) [noise=true , address=104.36.82.9:4021 , piece_id=T3KBEAMBTCD4I6UO52RTLVDFS47BRXJCZMGU44VEQITYQPM4K2YA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.920199227 (21h59m58.765000758s) [noise=true , address=104.36.82.50:4046 , piece_id=6UXAZY6MUTWLLVYZJBV4AYYIYH3KDR2IT22BWGGKBYJQGGMZGVIA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.920446438 (21h59m58.765247963s) [noise=true , address=77.90.31.19:4850 , piece_id=5FE4KIHO7B5CQ2ZHGB6CIZ4BA4ZYHMG666JB7S7YI3ZV25PM7DRQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.920500425 (21h59m58.765301948s) [noise=true , address=79.127.220.97:4367 , piece_id=466Y2CJB2PZ6EYNZLJAGIIXJHNAWS5P6L2GJXDWTYGA5CPVG2L3A, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.920773391 (21h59m58.765574929s) [noise=true , address=104.36.82.46:4509 , piece_id=LF6MNWECHJUQIIGTCYSQAMVWOUVYAJ7S7WPPANUYNDVVZPD26QCA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.920795516 (21h59m58.765597039s) [noise=true , address=79.127.219.47:4065 , piece_id=63NEETIH55O2QLL6OTFXXIGRE4MSCVSOSUKPNTCNHYENBJKPA5TQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.921069667 (21h59m58.76587119s) [noise=true , address=77.90.31.17:4790 , piece_id=PQGH3COTKNFS37FWGA6FLRVASZTELDOMRZVY5DGHTMVGZZIQMQWQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.921112088 (21h59m58.765913613s) [noise=true , address=79.127.201.215:4350 , piece_id=5V225ATIAYECLX2LXE2H5RZFJJCEAVTKIFCGKYWBOF532LOMGJJQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.921402936 (21h59m58.766204463s) [noise=true , address=45.91.116.7:4040 , piece_id=GFCCEQ5NJD4J7QZAXY7AEHZMCCRV3NHLNTK76YWW2TMOK46GKJQA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:16.026469477 (21h59m58.871270992s) [upload=21044]: Next returned nil segment Jul 25 09:35:16.026486428 (21h59m58.871287959s) [upload=21044]: Waiting for error group managing segments... Jul 25 09:35:16.032614751 (21h59m58.877416267s) [noise=true , address=77.90.31.19:4850 , piece_id=5FE4KIHO7B5CQ2ZHGB6CIZ4BA4ZYHMG666JB7S7YI3ZV25PM7DRQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.033106703 (21h59m58.877908219s) [noise=true , address=45.91.116.6:4994 , piece_id=JPWTQR5RKBZACY6XJDNQAEUCFDNYFLC7BYZCLHOZCCFJPXEKOSMA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.034162031 (21h59m58.87896354s) [noise=true , address=77.90.31.18:4830 , piece_id=2Z6ELONZADYWJJZPQUWKPTARWQI5D4CRE7XDIQBCUVVWAOUOYFGA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.036556969 (21h59m58.881358495s) [noise=true , address=104.36.82.46:4509 , piece_id=LF6MNWECHJUQIIGTCYSQAMVWOUVYAJ7S7WPPANUYNDVVZPD26QCA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.036598532 (21h59m58.881400048s) [noise=true , address=104.36.82.15:4284 , piece_id=3OMO32MXTLOB6JUNQQVR27UNL5CHYN5XWWG3TO3DUOOIMEWYYY3A, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.042836507 (21h59m58.887638032s) [noise=true , address=79.127.219.47:4065 , piece_id=63NEETIH55O2QLL6OTFXXIGRE4MSCVSOSUKPNTCNHYENBJKPA5TQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.046300094 (21h59m58.891101617s) [noise=true , address=77.90.31.14:4678 , piece_id=HIZS7J3DY6Q33PO2AI6UNXQ5S54L6DGG6ZAP3YKZZSEUINZCXEPQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.046770857 (21h59m58.891572381s) [noise=true , address=77.90.31.17:4790 , piece_id=PQGH3COTKNFS37FWGA6FLRVASZTELDOMRZVY5DGHTMVGZZIQMQWQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.047620050 (21h59m58.892421564s) [noise=true , address=104.36.82.9:4021 , piece_id=T3KBEAMBTCD4I6UO52RTLVDFS47BRXJCZMGU44VEQITYQPM4K2YA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.049041308 (21h59m58.893842842s) [noise=true , address=45.91.116.7:4040 , piece_id=GFCCEQ5NJD4J7QZAXY7AEHZMCCRV3NHLNTK76YWW2TMOK46GKJQA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.049699949 (21h59m58.894501458s) [noise=true , address=45.91.116.10:4099 , piece_id=CWLXVFK56DYPPB3ED5INVEHKBUPW6MSKZLAA7SAFQU3YFT3UNHXQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.058570793 (21h59m58.903372309s) [noise=true , address=104.36.82.50:4046 , piece_id=6UXAZY6MUTWLLVYZJBV4AYYIYH3KDR2IT22BWGGKBYJQGGMZGVIA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.063456832 (21h59m58.908258348s) [noise=true , address=77.90.31.16:4751 , piece_id=DCODR5LGROHG7DVIUSRLFRRBKR672JGYQ3JOOTJSLL2NVLY6L62Q, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.064332620 (21h59m58.909134133s) [noise=true , address=77.90.31.15:4720 , piece_id=FL7PFV4B2TZ54WTT3YYU3NPGT7RBNGGHRUS4QTOARDZASB7PKAGQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.064383494 (21h59m58.909185033s) [seg_pos={0 0}, upload=21044]: Segment reached optimal threshold of 14 pieces. Jul 25 09:35:16.064635168 (21h59m58.909436691s) [noise=true , address=79.127.220.97:4367 , piece_id=466Y2CJB2PZ6EYNZLJAGIIXJHNAWS5P6L2GJXDWTYGA5CPVG2L3A, seg_pos={0 0}, upload=21044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ukTBGQCf81aFiZhST9sqJEN28d1qg44gv27U2oPJKiGNirRPB): protocol: write tcp 159.223.143.39:49600->79.127.220.97:4367: use of closed network connection; write tcp 159.223.143.39:49600->79.127.220.97:4367: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:49600->79.127.220.97:4367: use of closed network connection Jul 25 09:35:16.064733953 (21h59m58.909535466s) [noise=true , address=79.127.205.235:4690 , piece_id=4MLADNII3WTUAMY5KKWUM2XSVFDR3NSLA32EMVJ3AIDOJENSF3SA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121SG1z7mpp74ipZ9b7hVE9WrEicBpRk84uoc3JW7FKupxKLcdg): protocol: read tcp 159.223.143.39:49260->79.127.205.235:4690: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 25 09:35:16.064916028 (21h59m58.90971754s) [noise=true , address=79.127.201.215:4350 , piece_id=5V225ATIAYECLX2LXE2H5RZFJJCEAVTKIFCGKYWBOF532LOMGJJQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WhZAWvHCTj5xVkUuVZtgkFB1dhFKpBPq18YMKkepFx5qxkUAr): protocol: read tcp 159.223.143.39:39812->79.127.201.215:4350: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 25 09:35:16.064952221 (21h59m58.909753734s) [noise=true , address=79.127.205.246:4263 , piece_id=2RB4RONQXXPKZIETGCVEOM2JQUC5EIKETQA6VKCSSHF6ARLG2BRQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZV35gnSB9uZZZQDr9v8esb7r4XH6TcxLdnC8LnESkoKxH2FZt): protocol: read tcp 159.223.143.39:54542->79.127.205.246:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 25 09:35:16.064963992 (21h59m58.909765516s) [seg_pos={0 0}, upload=21044]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 25 09:35:16.065052663 (21h59m58.909854191s) [upload=21044]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:35:16.065058796 (21h59m58.909860327s) [upload=21044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:35:16.170721369 (21h59m59.015522894s) [upload=21044]: Done uploading segments... Jul 25 09:35:16.170744580 (21h59m59.015546107s) [upload=21044]: Upload finished. err: <nil> Duration:* 780.80516ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 303.338486ms FAIL (2025-07-25 10:15:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 403, RequestID: 185574E861CBB3B7, HostID: , api error AccessDenied: Access Denied. Duration: 303.365318ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........XX................................................] duration: 541.884923ms FAIL (2025-07-25 10:15:07 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== oad=21019]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:40.996438590 (21h58m23.841240106s) [noise=true , address=77.90.31.13:4627 , piece_id=HBRLEALALTYTQHBRC3OM7VC2AN2D4GAXKF7M36TAF5HD7G6OTPBQ, seg_pos={0 0}, upload=21019]: Done uploading piece. err: <nil> Jul 25 09:33:40.996877017 (21h58m23.841678534s) [noise=true , address=77.90.31.17:4772 , piece_id=HZP632HLOH3CYRU3AZHCLZSOEX7BL4F22YGSO5NM6GY3YCIJ26PA, seg_pos={0 0}, upload=21019]: Done uploading piece. err: <nil> Jul 25 09:33:40.996900235 (21h58m23.841701752s) [seg_pos={0 0}, upload=21019]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jul 25 09:33:40.996968719 (21h58m23.841770241s) [upload=21019]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:33:40.996975079 (21h58m23.841776601s) [upload=21019]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:41.050252368 (21h58m23.895054131s) [upload=21019]: Done uploading segments... Jul 25 09:33:41.050275330 (21h58m23.895076853s) [upload=21019]: Upload finished. err: <nil> Jul 25 09:34:01.395455482 (21h58m44.240257015s) [upload=21028]: Starting upload Jul 25 09:34:01.395607427 (21h58m44.240408952s) [upload=21028]: Done starting upload Jul 25 09:34:01.395825566 (21h58m44.240627098s) [upload=21028]: Uploading segments... Jul 25 09:34:01.396495420 (21h58m44.241296955s) [upload=21028]: Got next segment. Inline: false Jul 25 09:34:01.396604095 (21h58m44.241405672s) [upload=21028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:34:01.396612168 (21h58m44.241413704s) [upload=21028]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:34:01.425826935 (21h58m44.270628471s) [seg_pos={0 0}, upload=21028]: Begin upload segment... Jul 25 09:34:01.426020430 (21h58m44.270821966s) [seg_pos={0 0}, upload=21028]: Done begin upload segment. Jul 25 09:34:01.426040728 (21h58m44.270842262s) [noise=true , address=104.36.82.4:4801 , piece_id=3NQQNM47LKMBO5GAQTMIWFTCGMD74JPMV4UPN7DTSVEKEZDZV5CA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.426061340 (21h58m44.2708629s) [noise=true , address=45.91.116.9:4263 , piece_id=VQVHJLCTXK5RMM2COSMGLP3UERPAFXL7CGCXROLSTIS4MIDTXGJA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.426601379 (21h58m44.271402918s) [noise=true , address=45.91.116.13:4155 , piece_id=V522D5F5OHME2REAEHIKBXKB4TA7FA24AOMZW5SMZV65UG4KW3XQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.426656014 (21h58m44.271457552s) [noise=true , address=77.90.31.13:4627 , piece_id=4AQ2PIDAK475LLLAG63ECCYHSZYWYVI52EJG7K5L7PA5BNBICA2A, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.427123147 (21h58m44.271924683s) [noise=true , address=45.91.116.7:4010 , piece_id=5OIKMU4MAE77EW635NMFJ7DOZNTCPC3QEYK52P5VITJJEZWFKEEA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.427565449 (21h58m44.272366988s) [noise=true , address=45.91.116.10:4083 , piece_id=D4PFM5EKLWNM5U3WVVHT5BVFXHPPNRFS64FEYLZORCR63FVQOEIQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.428074025 (21h58m44.272875567s) [noise=true , address=45.91.116.13:4154 , piece_id=OI6I6PDJKSYMDPFX4Q7AUWFRUXKMAZM7KI2X5QKTRHU3ZLYYCMVQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.428510546 (21h58m44.273312077s) [noise=true , address=77.90.31.16:4735 , piece_id=6ZY3FR55LO6O5E56LW6I5I5R7CHPC2MGS7NUJDD7UAXLOS3GXK4Q, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.428934503 (21h58m44.273736038s) [noise=true , address=45.91.116.6:4975 , piece_id=MI33ERQ7USAD2VE72XW5GV7G7HVUTDWTMUIXZQIBQQYOEHLR2DKQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.429488020 (21h58m44.274289548s) [noise=true , address=104.36.82.2:4730 , piece_id=VJNUI5MBLNWTMXZPXZNCFTSS4HBH7SBRXMQSMS36TIJOVLMA6X7Q, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.430016372 (21h58m44.274817913s) [noise=true , address=77.90.31.14:4661 , piece_id=AVBIPY5PRBUYUDVPMXCUFNUDQ6KPGDFMFW4HYZGUFLAKNC5ZLXYA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.430716875 (21h58m44.275518412s) [noise=true , address=104.36.82.12:4149 , piece_id=25LO6D4NKMKRLQQ4JNAXOLO5VAYANARND6RPNP5AW6J2WD5JCDXQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.431223998 (21h58m44.27602553s) [noise=true , address=77.90.31.19:4842 , piece_id=QWKQ455RQK2Q7IS3U5EXHAW3FX33XWJDTEV2GPDDAL5FJTEGTPGQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.431734863 (21h58m44.2765364s) [noise=true , address=104.36.82.14:4223 , piece_id=KG47KSNPFPH7PHYPLBC4GONV5XLYDG6M3HUMWZLK7G7PU6HPCXQQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.432299394 (21h58m44.277100922s) [noise=true , address=45.91.116.4:4937 , piece_id=GYYBC7JJCP2VOENJJAZRDAO6NMHVZXZEXA5M3APOATJCXDQ5XSNA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.432742241 (21h58m44.277543768s) [noise=true , address=77.90.31.15:4698 , piece_id=UKE2SEHPFX367ON5N2THRO5XN26KWOMQLP7RD36K2H5JTLXIRWOA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.541163047 (21h58m44.385964577s) [upload=21028]: Next returned nil segment Jul 25 09:34:01.541183693 (21h58m44.385985227s) [upload=21028]: Waiting for error group managing segments... Jul 25 09:34:01.555640486 (21h58m44.400441996s) [noise=true , address=45.91.116.6:4975 , piece_id=MI33ERQ7USAD2VE72XW5GV7G7HVUTDWTMUIXZQIBQQYOEHLR2DKQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.556377594 (21h58m44.401179119s) [noise=true , address=45.91.116.10:4083 , piece_id=D4PFM5EKLWNM5U3WVVHT5BVFXHPPNRFS64FEYLZORCR63FVQOEIQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.557556943 (21h58m44.402358454s) [noise=true , address=45.91.116.13:4155 , piece_id=V522D5F5OHME2REAEHIKBXKB4TA7FA24AOMZW5SMZV65UG4KW3XQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.558013379 (21h58m44.402814904s) [noise=true , address=77.90.31.13:4627 , piece_id=4AQ2PIDAK475LLLAG63ECCYHSZYWYVI52EJG7K5L7PA5BNBICA2A, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.558428945 (21h58m44.403230459s) [noise=true , address=45.91.116.9:4263 , piece_id=VQVHJLCTXK5RMM2COSMGLP3UERPAFXL7CGCXROLSTIS4MIDTXGJA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.558956623 (21h58m44.403758141s) [noise=true , address=104.36.82.2:4730 , piece_id=VJNUI5MBLNWTMXZPXZNCFTSS4HBH7SBRXMQSMS36TIJOVLMA6X7Q, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.559007198 (21h58m44.403808715s) [noise=true , address=45.91.116.13:4154 , piece_id=OI6I6PDJKSYMDPFX4Q7AUWFRUXKMAZM7KI2X5QKTRHU3ZLYYCMVQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.559590455 (21h58m44.404391965s) [noise=true , address=45.91.116.4:4937 , piece_id=GYYBC7JJCP2VOENJJAZRDAO6NMHVZXZEXA5M3APOATJCXDQ5XSNA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.559696005 (21h58m44.404497534s) [noise=true , address=77.90.31.14:4661 , piece_id=AVBIPY5PRBUYUDVPMXCUFNUDQ6KPGDFMFW4HYZGUFLAKNC5ZLXYA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.560471920 (21h58m44.405273433s) [noise=true , address=104.36.82.14:4223 , piece_id=KG47KSNPFPH7PHYPLBC4GONV5XLYDG6M3HUMWZLK7G7PU6HPCXQQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561137208 (21h58m44.405938736s) [noise=true , address=77.90.31.15:4698 , piece_id=UKE2SEHPFX367ON5N2THRO5XN26KWOMQLP7RD36K2H5JTLXIRWOA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561216161 (21h58m44.406017672s) [noise=true , address=77.90.31.19:4842 , piece_id=QWKQ455RQK2Q7IS3U5EXHAW3FX33XWJDTEV2GPDDAL5FJTEGTPGQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561662523 (21h58m44.40646404s) [noise=true , address=45.91.116.7:4010 , piece_id=5OIKMU4MAE77EW635NMFJ7DOZNTCPC3QEYK52P5VITJJEZWFKEEA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561675230 (21h58m44.406476756s) [seg_pos={0 0}, upload=21028]: Segment reached optimal threshold of 13 pieces. Jul 25 09:34:01.561800391 (21h58m44.406601905s) [noise=true , address=104.36.82.12:4149 , piece_id=25LO6D4NKMKRLQQ4JNAXOLO5VAYANARND6RPNP5AW6J2WD5JCDXQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:01.561994528 (21h58m44.406796058s) [noise=true , address=104.36.82.4:4801 , piece_id=3NQQNM47LKMBO5GAQTMIWFTCGMD74JPMV4UPN7DTSVEKEZDZV5CA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:01.562128894 (21h58m44.406930406s) [noise=true , address=77.90.31.16:4735 , piece_id=6ZY3FR55LO6O5E56LW6I5I5R7CHPC2MGS7NUJDD7UAXLOS3GXK4Q, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.562229276 (21h58m44.407030809s) [seg_pos={0 0}, upload=21028]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 25 09:34:01.562358118 (21h58m44.407159707s) [upload=21028]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:34:01.562365734 (21h58m44.407167258s) [upload=21028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:01.612443727 (21h58m44.457245259s) [upload=21028]: Done uploading segments... Jul 25 09:34:01.612489093 (21h58m44.457290622s) [upload=21028]: Upload finished. err: <nil> Duration: 541.917114ms core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 43.119537ms remote: "tcp:45.38.24.2:443" response headers: Content-Length=279 Content-Security-Policy=block-all-mixed-content X-Request-Id=D5AmxJ2hsti X-Xss-Protection=1; mode=block Date=Fri, 25 Jul 2025 09:36:06 GMT Accept-Ranges=bytes Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=185574E451263CD8 target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-07-25 10:15:07 UTC) orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Access Denied.</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185574E451263CD8</RequestId><HostId></HostId></Error>" Duration: 43.169332ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 542.296551ms FAIL (2025-07-25 10:14:54 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 542.329016ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......XX...................................................] duration: 1.736853559s FAIL (2025-07-25 10:14:46 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 1.73689383s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 506.973316ms FAIL (2025-07-25 10:14:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 506.999759ms edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 50.633418ms remote: "tcp:45.38.24.2:443" response headers: Content-Length=279 Content-Security-Policy=block-all-mixed-content X-Request-Id=D5AmxJ2hsti X-Xss-Protection=1; mode=block Date=Fri, 25 Jul 2025 09:36:06 GMT Accept-Ranges=bytes Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=185574E451263CD8 target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-07-25 10:14:30 UTC) orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Access Denied.</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185574E451263CD8</RequestId><HostId></HostId></Error>" Duration: 50.662486ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....XXX....................................................] duration: 477.967618ms FAIL (2025-07-25 10:14:26 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 478.113326ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [........X............X...X..................................] duration: 275.132575ms FAIL (2025-07-25 10:14:25 UTC) orbiter warning: uplink: invalid range 0:1024 (size:1024) Duration: 275.161446ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........XX...............................................] duration: 592.915907ms FAIL (2025-07-25 10:14:23 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== oad=21019]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:40.996438590 (21h58m23.841240106s) [noise=true , address=77.90.31.13:4627 , piece_id=HBRLEALALTYTQHBRC3OM7VC2AN2D4GAXKF7M36TAF5HD7G6OTPBQ, seg_pos={0 0}, upload=21019]: Done uploading piece. err: <nil> Jul 25 09:33:40.996877017 (21h58m23.841678534s) [noise=true , address=77.90.31.17:4772 , piece_id=HZP632HLOH3CYRU3AZHCLZSOEX7BL4F22YGSO5NM6GY3YCIJ26PA, seg_pos={0 0}, upload=21019]: Done uploading piece. err: <nil> Jul 25 09:33:40.996900235 (21h58m23.841701752s) [seg_pos={0 0}, upload=21019]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jul 25 09:33:40.996968719 (21h58m23.841770241s) [upload=21019]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:33:40.996975079 (21h58m23.841776601s) [upload=21019]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:41.050252368 (21h58m23.895054131s) [upload=21019]: Done uploading segments... Jul 25 09:33:41.050275330 (21h58m23.895076853s) [upload=21019]: Upload finished. err: <nil> Jul 25 09:34:01.395455482 (21h58m44.240257015s) [upload=21028]: Starting upload Jul 25 09:34:01.395607427 (21h58m44.240408952s) [upload=21028]: Done starting upload Jul 25 09:34:01.395825566 (21h58m44.240627098s) [upload=21028]: Uploading segments... Jul 25 09:34:01.396495420 (21h58m44.241296955s) [upload=21028]: Got next segment. Inline: false Jul 25 09:34:01.396604095 (21h58m44.241405672s) [upload=21028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:34:01.396612168 (21h58m44.241413704s) [upload=21028]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:34:01.425826935 (21h58m44.270628471s) [seg_pos={0 0}, upload=21028]: Begin upload segment... Jul 25 09:34:01.426020430 (21h58m44.270821966s) [seg_pos={0 0}, upload=21028]: Done begin upload segment. Jul 25 09:34:01.426040728 (21h58m44.270842262s) [noise=true , address=104.36.82.4:4801 , piece_id=3NQQNM47LKMBO5GAQTMIWFTCGMD74JPMV4UPN7DTSVEKEZDZV5CA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.426061340 (21h58m44.2708629s) [noise=true , address=45.91.116.9:4263 , piece_id=VQVHJLCTXK5RMM2COSMGLP3UERPAFXL7CGCXROLSTIS4MIDTXGJA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.426601379 (21h58m44.271402918s) [noise=true , address=45.91.116.13:4155 , piece_id=V522D5F5OHME2REAEHIKBXKB4TA7FA24AOMZW5SMZV65UG4KW3XQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.426656014 (21h58m44.271457552s) [noise=true , address=77.90.31.13:4627 , piece_id=4AQ2PIDAK475LLLAG63ECCYHSZYWYVI52EJG7K5L7PA5BNBICA2A, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.427123147 (21h58m44.271924683s) [noise=true , address=45.91.116.7:4010 , piece_id=5OIKMU4MAE77EW635NMFJ7DOZNTCPC3QEYK52P5VITJJEZWFKEEA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.427565449 (21h58m44.272366988s) [noise=true , address=45.91.116.10:4083 , piece_id=D4PFM5EKLWNM5U3WVVHT5BVFXHPPNRFS64FEYLZORCR63FVQOEIQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.428074025 (21h58m44.272875567s) [noise=true , address=45.91.116.13:4154 , piece_id=OI6I6PDJKSYMDPFX4Q7AUWFRUXKMAZM7KI2X5QKTRHU3ZLYYCMVQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.428510546 (21h58m44.273312077s) [noise=true , address=77.90.31.16:4735 , piece_id=6ZY3FR55LO6O5E56LW6I5I5R7CHPC2MGS7NUJDD7UAXLOS3GXK4Q, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.428934503 (21h58m44.273736038s) [noise=true , address=45.91.116.6:4975 , piece_id=MI33ERQ7USAD2VE72XW5GV7G7HVUTDWTMUIXZQIBQQYOEHLR2DKQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.429488020 (21h58m44.274289548s) [noise=true , address=104.36.82.2:4730 , piece_id=VJNUI5MBLNWTMXZPXZNCFTSS4HBH7SBRXMQSMS36TIJOVLMA6X7Q, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.430016372 (21h58m44.274817913s) [noise=true , address=77.90.31.14:4661 , piece_id=AVBIPY5PRBUYUDVPMXCUFNUDQ6KPGDFMFW4HYZGUFLAKNC5ZLXYA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.430716875 (21h58m44.275518412s) [noise=true , address=104.36.82.12:4149 , piece_id=25LO6D4NKMKRLQQ4JNAXOLO5VAYANARND6RPNP5AW6J2WD5JCDXQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.431223998 (21h58m44.27602553s) [noise=true , address=77.90.31.19:4842 , piece_id=QWKQ455RQK2Q7IS3U5EXHAW3FX33XWJDTEV2GPDDAL5FJTEGTPGQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.431734863 (21h58m44.2765364s) [noise=true , address=104.36.82.14:4223 , piece_id=KG47KSNPFPH7PHYPLBC4GONV5XLYDG6M3HUMWZLK7G7PU6HPCXQQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.432299394 (21h58m44.277100922s) [noise=true , address=45.91.116.4:4937 , piece_id=GYYBC7JJCP2VOENJJAZRDAO6NMHVZXZEXA5M3APOATJCXDQ5XSNA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.432742241 (21h58m44.277543768s) [noise=true , address=77.90.31.15:4698 , piece_id=UKE2SEHPFX367ON5N2THRO5XN26KWOMQLP7RD36K2H5JTLXIRWOA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.541163047 (21h58m44.385964577s) [upload=21028]: Next returned nil segment Jul 25 09:34:01.541183693 (21h58m44.385985227s) [upload=21028]: Waiting for error group managing segments... Jul 25 09:34:01.555640486 (21h58m44.400441996s) [noise=true , address=45.91.116.6:4975 , piece_id=MI33ERQ7USAD2VE72XW5GV7G7HVUTDWTMUIXZQIBQQYOEHLR2DKQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.556377594 (21h58m44.401179119s) [noise=true , address=45.91.116.10:4083 , piece_id=D4PFM5EKLWNM5U3WVVHT5BVFXHPPNRFS64FEYLZORCR63FVQOEIQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.557556943 (21h58m44.402358454s) [noise=true , address=45.91.116.13:4155 , piece_id=V522D5F5OHME2REAEHIKBXKB4TA7FA24AOMZW5SMZV65UG4KW3XQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.558013379 (21h58m44.402814904s) [noise=true , address=77.90.31.13:4627 , piece_id=4AQ2PIDAK475LLLAG63ECCYHSZYWYVI52EJG7K5L7PA5BNBICA2A, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.558428945 (21h58m44.403230459s) [noise=true , address=45.91.116.9:4263 , piece_id=VQVHJLCTXK5RMM2COSMGLP3UERPAFXL7CGCXROLSTIS4MIDTXGJA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.558956623 (21h58m44.403758141s) [noise=true , address=104.36.82.2:4730 , piece_id=VJNUI5MBLNWTMXZPXZNCFTSS4HBH7SBRXMQSMS36TIJOVLMA6X7Q, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.559007198 (21h58m44.403808715s) [noise=true , address=45.91.116.13:4154 , piece_id=OI6I6PDJKSYMDPFX4Q7AUWFRUXKMAZM7KI2X5QKTRHU3ZLYYCMVQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.559590455 (21h58m44.404391965s) [noise=true , address=45.91.116.4:4937 , piece_id=GYYBC7JJCP2VOENJJAZRDAO6NMHVZXZEXA5M3APOATJCXDQ5XSNA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.559696005 (21h58m44.404497534s) [noise=true , address=77.90.31.14:4661 , piece_id=AVBIPY5PRBUYUDVPMXCUFNUDQ6KPGDFMFW4HYZGUFLAKNC5ZLXYA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.560471920 (21h58m44.405273433s) [noise=true , address=104.36.82.14:4223 , piece_id=KG47KSNPFPH7PHYPLBC4GONV5XLYDG6M3HUMWZLK7G7PU6HPCXQQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561137208 (21h58m44.405938736s) [noise=true , address=77.90.31.15:4698 , piece_id=UKE2SEHPFX367ON5N2THRO5XN26KWOMQLP7RD36K2H5JTLXIRWOA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561216161 (21h58m44.406017672s) [noise=true , address=77.90.31.19:4842 , piece_id=QWKQ455RQK2Q7IS3U5EXHAW3FX33XWJDTEV2GPDDAL5FJTEGTPGQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561662523 (21h58m44.40646404s) [noise=true , address=45.91.116.7:4010 , piece_id=5OIKMU4MAE77EW635NMFJ7DOZNTCPC3QEYK52P5VITJJEZWFKEEA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561675230 (21h58m44.406476756s) [seg_pos={0 0}, upload=21028]: Segment reached optimal threshold of 13 pieces. Jul 25 09:34:01.561800391 (21h58m44.406601905s) [noise=true , address=104.36.82.12:4149 , piece_id=25LO6D4NKMKRLQQ4JNAXOLO5VAYANARND6RPNP5AW6J2WD5JCDXQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:01.561994528 (21h58m44.406796058s) [noise=true , address=104.36.82.4:4801 , piece_id=3NQQNM47LKMBO5GAQTMIWFTCGMD74JPMV4UPN7DTSVEKEZDZV5CA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:01.562128894 (21h58m44.406930406s) [noise=true , address=77.90.31.16:4735 , piece_id=6ZY3FR55LO6O5E56LW6I5I5R7CHPC2MGS7NUJDD7UAXLOS3GXK4Q, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.562229276 (21h58m44.407030809s) [seg_pos={0 0}, upload=21028]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 25 09:34:01.562358118 (21h58m44.407159707s) [upload=21028]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:34:01.562365734 (21h58m44.407167258s) [upload=21028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:01.612443727 (21h58m44.457245259s) [upload=21028]: Done uploading segments... Jul 25 09:34:01.612489093 (21h58m44.457290622s) [upload=21028]: Upload finished. err: <nil> Duration: 592.961461ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 624.065082ms FAIL (2025-07-25 10:14:17 UTC) orbiter warning: ======ERROR====== uplink: permission denied (bucket: metaclient: Unauthorized API credentials) ======LOGS====== h item: pb.BatchRequestItem_SegmentBegin Jul 25 09:35:15.918314780 (21h59m58.763116293s) [seg_pos={0 0}, upload=21044]: Begin upload segment... Jul 25 09:35:15.918358070 (21h59m58.763159586s) [seg_pos={0 0}, upload=21044]: Done begin upload segment. Jul 25 09:35:15.918382251 (21h59m58.763183776s) [noise=true , address=77.90.31.14:4678 , piece_id=HIZS7J3DY6Q33PO2AI6UNXQ5S54L6DGG6ZAP3YKZZSEUINZCXEPQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.918407152 (21h59m58.763208685s) [noise=true , address=45.91.116.6:4994 , piece_id=JPWTQR5RKBZACY6XJDNQAEUCFDNYFLC7BYZCLHOZCCFJPXEKOSMA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.918842345 (21h59m58.763643878s) [noise=true , address=77.90.31.16:4751 , piece_id=DCODR5LGROHG7DVIUSRLFRRBKR672JGYQ3JOOTJSLL2NVLY6L62Q, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.918965591 (21h59m58.763767123s) [noise=true , address=77.90.31.15:4720 , piece_id=FL7PFV4B2TZ54WTT3YYU3NPGT7RBNGGHRUS4QTOARDZASB7PKAGQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.919180183 (21h59m58.763981708s) [noise=true , address=77.90.31.18:4830 , piece_id=2Z6ELONZADYWJJZPQUWKPTARWQI5D4CRE7XDIQBCUVVWAOUOYFGA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.919362724 (21h59m58.76416425s) [noise=true , address=79.127.205.235:4690 , piece_id=4MLADNII3WTUAMY5KKWUM2XSVFDR3NSLA32EMVJ3AIDOJENSF3SA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.919457169 (21h59m58.764258696s) [noise=true , address=79.127.205.246:4263 , piece_id=2RB4RONQXXPKZIETGCVEOM2JQUC5EIKETQA6VKCSSHF6ARLG2BRQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.919677786 (21h59m58.764479318s) [noise=true , address=104.36.82.15:4284 , piece_id=3OMO32MXTLOB6JUNQQVR27UNL5CHYN5XWWG3TO3DUOOIMEWYYY3A, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.919849288 (21h59m58.764650816s) [noise=true , address=45.91.116.10:4099 , piece_id=CWLXVFK56DYPPB3ED5INVEHKBUPW6MSKZLAA7SAFQU3YFT3UNHXQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.920171246 (21h59m58.764972769s) [noise=true , address=104.36.82.9:4021 , piece_id=T3KBEAMBTCD4I6UO52RTLVDFS47BRXJCZMGU44VEQITYQPM4K2YA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.920199227 (21h59m58.765000758s) [noise=true , address=104.36.82.50:4046 , piece_id=6UXAZY6MUTWLLVYZJBV4AYYIYH3KDR2IT22BWGGKBYJQGGMZGVIA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.920446438 (21h59m58.765247963s) [noise=true , address=77.90.31.19:4850 , piece_id=5FE4KIHO7B5CQ2ZHGB6CIZ4BA4ZYHMG666JB7S7YI3ZV25PM7DRQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.920500425 (21h59m58.765301948s) [noise=true , address=79.127.220.97:4367 , piece_id=466Y2CJB2PZ6EYNZLJAGIIXJHNAWS5P6L2GJXDWTYGA5CPVG2L3A, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.920773391 (21h59m58.765574929s) [noise=true , address=104.36.82.46:4509 , piece_id=LF6MNWECHJUQIIGTCYSQAMVWOUVYAJ7S7WPPANUYNDVVZPD26QCA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.920795516 (21h59m58.765597039s) [noise=true , address=79.127.219.47:4065 , piece_id=63NEETIH55O2QLL6OTFXXIGRE4MSCVSOSUKPNTCNHYENBJKPA5TQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.921069667 (21h59m58.76587119s) [noise=true , address=77.90.31.17:4790 , piece_id=PQGH3COTKNFS37FWGA6FLRVASZTELDOMRZVY5DGHTMVGZZIQMQWQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.921112088 (21h59m58.765913613s) [noise=true , address=79.127.201.215:4350 , piece_id=5V225ATIAYECLX2LXE2H5RZFJJCEAVTKIFCGKYWBOF532LOMGJJQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.921402936 (21h59m58.766204463s) [noise=true , address=45.91.116.7:4040 , piece_id=GFCCEQ5NJD4J7QZAXY7AEHZMCCRV3NHLNTK76YWW2TMOK46GKJQA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:16.026469477 (21h59m58.871270992s) [upload=21044]: Next returned nil segment Jul 25 09:35:16.026486428 (21h59m58.871287959s) [upload=21044]: Waiting for error group managing segments... Jul 25 09:35:16.032614751 (21h59m58.877416267s) [noise=true , address=77.90.31.19:4850 , piece_id=5FE4KIHO7B5CQ2ZHGB6CIZ4BA4ZYHMG666JB7S7YI3ZV25PM7DRQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.033106703 (21h59m58.877908219s) [noise=true , address=45.91.116.6:4994 , piece_id=JPWTQR5RKBZACY6XJDNQAEUCFDNYFLC7BYZCLHOZCCFJPXEKOSMA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.034162031 (21h59m58.87896354s) [noise=true , address=77.90.31.18:4830 , piece_id=2Z6ELONZADYWJJZPQUWKPTARWQI5D4CRE7XDIQBCUVVWAOUOYFGA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.036556969 (21h59m58.881358495s) [noise=true , address=104.36.82.46:4509 , piece_id=LF6MNWECHJUQIIGTCYSQAMVWOUVYAJ7S7WPPANUYNDVVZPD26QCA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.036598532 (21h59m58.881400048s) [noise=true , address=104.36.82.15:4284 , piece_id=3OMO32MXTLOB6JUNQQVR27UNL5CHYN5XWWG3TO3DUOOIMEWYYY3A, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.042836507 (21h59m58.887638032s) [noise=true , address=79.127.219.47:4065 , piece_id=63NEETIH55O2QLL6OTFXXIGRE4MSCVSOSUKPNTCNHYENBJKPA5TQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.046300094 (21h59m58.891101617s) [noise=true , address=77.90.31.14:4678 , piece_id=HIZS7J3DY6Q33PO2AI6UNXQ5S54L6DGG6ZAP3YKZZSEUINZCXEPQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.046770857 (21h59m58.891572381s) [noise=true , address=77.90.31.17:4790 , piece_id=PQGH3COTKNFS37FWGA6FLRVASZTELDOMRZVY5DGHTMVGZZIQMQWQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.047620050 (21h59m58.892421564s) [noise=true , address=104.36.82.9:4021 , piece_id=T3KBEAMBTCD4I6UO52RTLVDFS47BRXJCZMGU44VEQITYQPM4K2YA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.049041308 (21h59m58.893842842s) [noise=true , address=45.91.116.7:4040 , piece_id=GFCCEQ5NJD4J7QZAXY7AEHZMCCRV3NHLNTK76YWW2TMOK46GKJQA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.049699949 (21h59m58.894501458s) [noise=true , address=45.91.116.10:4099 , piece_id=CWLXVFK56DYPPB3ED5INVEHKBUPW6MSKZLAA7SAFQU3YFT3UNHXQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.058570793 (21h59m58.903372309s) [noise=true , address=104.36.82.50:4046 , piece_id=6UXAZY6MUTWLLVYZJBV4AYYIYH3KDR2IT22BWGGKBYJQGGMZGVIA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.063456832 (21h59m58.908258348s) [noise=true , address=77.90.31.16:4751 , piece_id=DCODR5LGROHG7DVIUSRLFRRBKR672JGYQ3JOOTJSLL2NVLY6L62Q, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.064332620 (21h59m58.909134133s) [noise=true , address=77.90.31.15:4720 , piece_id=FL7PFV4B2TZ54WTT3YYU3NPGT7RBNGGHRUS4QTOARDZASB7PKAGQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.064383494 (21h59m58.909185033s) [seg_pos={0 0}, upload=21044]: Segment reached optimal threshold of 14 pieces. Jul 25 09:35:16.064635168 (21h59m58.909436691s) [noise=true , address=79.127.220.97:4367 , piece_id=466Y2CJB2PZ6EYNZLJAGIIXJHNAWS5P6L2GJXDWTYGA5CPVG2L3A, seg_pos={0 0}, upload=21044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ukTBGQCf81aFiZhST9sqJEN28d1qg44gv27U2oPJKiGNirRPB): protocol: write tcp 159.223.143.39:49600->79.127.220.97:4367: use of closed network connection; write tcp 159.223.143.39:49600->79.127.220.97:4367: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:49600->79.127.220.97:4367: use of closed network connection Jul 25 09:35:16.064733953 (21h59m58.909535466s) [noise=true , address=79.127.205.235:4690 , piece_id=4MLADNII3WTUAMY5KKWUM2XSVFDR3NSLA32EMVJ3AIDOJENSF3SA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121SG1z7mpp74ipZ9b7hVE9WrEicBpRk84uoc3JW7FKupxKLcdg): protocol: read tcp 159.223.143.39:49260->79.127.205.235:4690: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 25 09:35:16.064916028 (21h59m58.90971754s) [noise=true , address=79.127.201.215:4350 , piece_id=5V225ATIAYECLX2LXE2H5RZFJJCEAVTKIFCGKYWBOF532LOMGJJQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WhZAWvHCTj5xVkUuVZtgkFB1dhFKpBPq18YMKkepFx5qxkUAr): protocol: read tcp 159.223.143.39:39812->79.127.201.215:4350: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 25 09:35:16.064952221 (21h59m58.909753734s) [noise=true , address=79.127.205.246:4263 , piece_id=2RB4RONQXXPKZIETGCVEOM2JQUC5EIKETQA6VKCSSHF6ARLG2BRQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZV35gnSB9uZZZQDr9v8esb7r4XH6TcxLdnC8LnESkoKxH2FZt): protocol: read tcp 159.223.143.39:54542->79.127.205.246:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 25 09:35:16.064963992 (21h59m58.909765516s) [seg_pos={0 0}, upload=21044]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 25 09:35:16.065052663 (21h59m58.909854191s) [upload=21044]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:35:16.065058796 (21h59m58.909860327s) [upload=21044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:35:16.170721369 (21h59m59.015522894s) [upload=21044]: Done uploading segments... Jul 25 09:35:16.170744580 (21h59m59.015546107s) [upload=21044]: Upload finished. err: <nil> Duration:* 624.104063ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 35.685159ms remote: "tcp:45.38.24.2:443" response headers: Content-Length=279 Content-Security-Policy=block-all-mixed-content X-Request-Id=D5AmxJ2hsti X-Xss-Protection=1; mode=block Date=Fri, 25 Jul 2025 09:36:06 GMT Accept-Ranges=bytes Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=185574E451263CD8 target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-07-25 10:14:10 UTC) orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Access Denied.</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185574E451263CD8</RequestId><HostId></HostId></Error>" Duration: 35.742684ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........XX..................................................] duration: 1.796751093s FAIL (2025-07-25 10:14:09 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 1.796791358s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 563.088298ms FAIL (2025-07-25 10:14:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 403, RequestID: 185574E861CBB3B7, HostID: , api error AccessDenied: Access Denied. Duration: 563.133617ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [......XXX...................................................] duration: 431.113234ms FAIL (2025-07-25 10:14:02 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 431.203371ms core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 518.058953ms FAIL (2025-07-25 10:14:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 518.086428ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 461.779319ms FAIL (2025-07-25 10:13:49 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 461.829326ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 459.584469ms FAIL (2025-07-25 10:13:34 UTC) orbiter warning: ======ERROR====== uplink: permission denied (bucket: metaclient: Unauthorized API credentials) ======LOGS====== h item: pb.BatchRequestItem_SegmentBegin Jul 25 09:35:15.918314780 (21h59m58.763116293s) [seg_pos={0 0}, upload=21044]: Begin upload segment... Jul 25 09:35:15.918358070 (21h59m58.763159586s) [seg_pos={0 0}, upload=21044]: Done begin upload segment. Jul 25 09:35:15.918382251 (21h59m58.763183776s) [noise=true , address=77.90.31.14:4678 , piece_id=HIZS7J3DY6Q33PO2AI6UNXQ5S54L6DGG6ZAP3YKZZSEUINZCXEPQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.918407152 (21h59m58.763208685s) [noise=true , address=45.91.116.6:4994 , piece_id=JPWTQR5RKBZACY6XJDNQAEUCFDNYFLC7BYZCLHOZCCFJPXEKOSMA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.918842345 (21h59m58.763643878s) [noise=true , address=77.90.31.16:4751 , piece_id=DCODR5LGROHG7DVIUSRLFRRBKR672JGYQ3JOOTJSLL2NVLY6L62Q, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.918965591 (21h59m58.763767123s) [noise=true , address=77.90.31.15:4720 , piece_id=FL7PFV4B2TZ54WTT3YYU3NPGT7RBNGGHRUS4QTOARDZASB7PKAGQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.919180183 (21h59m58.763981708s) [noise=true , address=77.90.31.18:4830 , piece_id=2Z6ELONZADYWJJZPQUWKPTARWQI5D4CRE7XDIQBCUVVWAOUOYFGA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.919362724 (21h59m58.76416425s) [noise=true , address=79.127.205.235:4690 , piece_id=4MLADNII3WTUAMY5KKWUM2XSVFDR3NSLA32EMVJ3AIDOJENSF3SA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.919457169 (21h59m58.764258696s) [noise=true , address=79.127.205.246:4263 , piece_id=2RB4RONQXXPKZIETGCVEOM2JQUC5EIKETQA6VKCSSHF6ARLG2BRQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.919677786 (21h59m58.764479318s) [noise=true , address=104.36.82.15:4284 , piece_id=3OMO32MXTLOB6JUNQQVR27UNL5CHYN5XWWG3TO3DUOOIMEWYYY3A, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.919849288 (21h59m58.764650816s) [noise=true , address=45.91.116.10:4099 , piece_id=CWLXVFK56DYPPB3ED5INVEHKBUPW6MSKZLAA7SAFQU3YFT3UNHXQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.920171246 (21h59m58.764972769s) [noise=true , address=104.36.82.9:4021 , piece_id=T3KBEAMBTCD4I6UO52RTLVDFS47BRXJCZMGU44VEQITYQPM4K2YA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.920199227 (21h59m58.765000758s) [noise=true , address=104.36.82.50:4046 , piece_id=6UXAZY6MUTWLLVYZJBV4AYYIYH3KDR2IT22BWGGKBYJQGGMZGVIA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.920446438 (21h59m58.765247963s) [noise=true , address=77.90.31.19:4850 , piece_id=5FE4KIHO7B5CQ2ZHGB6CIZ4BA4ZYHMG666JB7S7YI3ZV25PM7DRQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.920500425 (21h59m58.765301948s) [noise=true , address=79.127.220.97:4367 , piece_id=466Y2CJB2PZ6EYNZLJAGIIXJHNAWS5P6L2GJXDWTYGA5CPVG2L3A, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.920773391 (21h59m58.765574929s) [noise=true , address=104.36.82.46:4509 , piece_id=LF6MNWECHJUQIIGTCYSQAMVWOUVYAJ7S7WPPANUYNDVVZPD26QCA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.920795516 (21h59m58.765597039s) [noise=true , address=79.127.219.47:4065 , piece_id=63NEETIH55O2QLL6OTFXXIGRE4MSCVSOSUKPNTCNHYENBJKPA5TQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.921069667 (21h59m58.76587119s) [noise=true , address=77.90.31.17:4790 , piece_id=PQGH3COTKNFS37FWGA6FLRVASZTELDOMRZVY5DGHTMVGZZIQMQWQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.921112088 (21h59m58.765913613s) [noise=true , address=79.127.201.215:4350 , piece_id=5V225ATIAYECLX2LXE2H5RZFJJCEAVTKIFCGKYWBOF532LOMGJJQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.921402936 (21h59m58.766204463s) [noise=true , address=45.91.116.7:4040 , piece_id=GFCCEQ5NJD4J7QZAXY7AEHZMCCRV3NHLNTK76YWW2TMOK46GKJQA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:16.026469477 (21h59m58.871270992s) [upload=21044]: Next returned nil segment Jul 25 09:35:16.026486428 (21h59m58.871287959s) [upload=21044]: Waiting for error group managing segments... Jul 25 09:35:16.032614751 (21h59m58.877416267s) [noise=true , address=77.90.31.19:4850 , piece_id=5FE4KIHO7B5CQ2ZHGB6CIZ4BA4ZYHMG666JB7S7YI3ZV25PM7DRQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.033106703 (21h59m58.877908219s) [noise=true , address=45.91.116.6:4994 , piece_id=JPWTQR5RKBZACY6XJDNQAEUCFDNYFLC7BYZCLHOZCCFJPXEKOSMA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.034162031 (21h59m58.87896354s) [noise=true , address=77.90.31.18:4830 , piece_id=2Z6ELONZADYWJJZPQUWKPTARWQI5D4CRE7XDIQBCUVVWAOUOYFGA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.036556969 (21h59m58.881358495s) [noise=true , address=104.36.82.46:4509 , piece_id=LF6MNWECHJUQIIGTCYSQAMVWOUVYAJ7S7WPPANUYNDVVZPD26QCA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.036598532 (21h59m58.881400048s) [noise=true , address=104.36.82.15:4284 , piece_id=3OMO32MXTLOB6JUNQQVR27UNL5CHYN5XWWG3TO3DUOOIMEWYYY3A, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.042836507 (21h59m58.887638032s) [noise=true , address=79.127.219.47:4065 , piece_id=63NEETIH55O2QLL6OTFXXIGRE4MSCVSOSUKPNTCNHYENBJKPA5TQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.046300094 (21h59m58.891101617s) [noise=true , address=77.90.31.14:4678 , piece_id=HIZS7J3DY6Q33PO2AI6UNXQ5S54L6DGG6ZAP3YKZZSEUINZCXEPQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.046770857 (21h59m58.891572381s) [noise=true , address=77.90.31.17:4790 , piece_id=PQGH3COTKNFS37FWGA6FLRVASZTELDOMRZVY5DGHTMVGZZIQMQWQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.047620050 (21h59m58.892421564s) [noise=true , address=104.36.82.9:4021 , piece_id=T3KBEAMBTCD4I6UO52RTLVDFS47BRXJCZMGU44VEQITYQPM4K2YA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.049041308 (21h59m58.893842842s) [noise=true , address=45.91.116.7:4040 , piece_id=GFCCEQ5NJD4J7QZAXY7AEHZMCCRV3NHLNTK76YWW2TMOK46GKJQA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.049699949 (21h59m58.894501458s) [noise=true , address=45.91.116.10:4099 , piece_id=CWLXVFK56DYPPB3ED5INVEHKBUPW6MSKZLAA7SAFQU3YFT3UNHXQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.058570793 (21h59m58.903372309s) [noise=true , address=104.36.82.50:4046 , piece_id=6UXAZY6MUTWLLVYZJBV4AYYIYH3KDR2IT22BWGGKBYJQGGMZGVIA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.063456832 (21h59m58.908258348s) [noise=true , address=77.90.31.16:4751 , piece_id=DCODR5LGROHG7DVIUSRLFRRBKR672JGYQ3JOOTJSLL2NVLY6L62Q, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.064332620 (21h59m58.909134133s) [noise=true , address=77.90.31.15:4720 , piece_id=FL7PFV4B2TZ54WTT3YYU3NPGT7RBNGGHRUS4QTOARDZASB7PKAGQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.064383494 (21h59m58.909185033s) [seg_pos={0 0}, upload=21044]: Segment reached optimal threshold of 14 pieces. Jul 25 09:35:16.064635168 (21h59m58.909436691s) [noise=true , address=79.127.220.97:4367 , piece_id=466Y2CJB2PZ6EYNZLJAGIIXJHNAWS5P6L2GJXDWTYGA5CPVG2L3A, seg_pos={0 0}, upload=21044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ukTBGQCf81aFiZhST9sqJEN28d1qg44gv27U2oPJKiGNirRPB): protocol: write tcp 159.223.143.39:49600->79.127.220.97:4367: use of closed network connection; write tcp 159.223.143.39:49600->79.127.220.97:4367: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:49600->79.127.220.97:4367: use of closed network connection Jul 25 09:35:16.064733953 (21h59m58.909535466s) [noise=true , address=79.127.205.235:4690 , piece_id=4MLADNII3WTUAMY5KKWUM2XSVFDR3NSLA32EMVJ3AIDOJENSF3SA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121SG1z7mpp74ipZ9b7hVE9WrEicBpRk84uoc3JW7FKupxKLcdg): protocol: read tcp 159.223.143.39:49260->79.127.205.235:4690: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 25 09:35:16.064916028 (21h59m58.90971754s) [noise=true , address=79.127.201.215:4350 , piece_id=5V225ATIAYECLX2LXE2H5RZFJJCEAVTKIFCGKYWBOF532LOMGJJQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WhZAWvHCTj5xVkUuVZtgkFB1dhFKpBPq18YMKkepFx5qxkUAr): protocol: read tcp 159.223.143.39:39812->79.127.201.215:4350: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 25 09:35:16.064952221 (21h59m58.909753734s) [noise=true , address=79.127.205.246:4263 , piece_id=2RB4RONQXXPKZIETGCVEOM2JQUC5EIKETQA6VKCSSHF6ARLG2BRQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZV35gnSB9uZZZQDr9v8esb7r4XH6TcxLdnC8LnESkoKxH2FZt): protocol: read tcp 159.223.143.39:54542->79.127.205.246:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 25 09:35:16.064963992 (21h59m58.909765516s) [seg_pos={0 0}, upload=21044]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 25 09:35:16.065052663 (21h59m58.909854191s) [upload=21044]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:35:16.065058796 (21h59m58.909860327s) [upload=21044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:35:16.170721369 (21h59m59.015522894s) [upload=21044]: Done uploading segments... Jul 25 09:35:16.170744580 (21h59m59.015546107s) [upload=21044]: Upload finished. err: <nil> Duration:* 459.658606ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......XXX..................................................] duration: 454.29642ms FAIL (2025-07-25 10:13:32 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 454.326736ms core uplink us1 vivint
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 47.055916ms remote: "tcp:45.38.24.2:443" response headers: Content-Length=279 Content-Security-Policy=block-all-mixed-content X-Request-Id=D5AmxJ2hsti X-Xss-Protection=1; mode=block Date=Fri, 25 Jul 2025 09:36:06 GMT Accept-Ranges=bytes Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=185574E451263CD8 target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-07-25 10:13:29 UTC) orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Access Denied.</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185574E451263CD8</RequestId><HostId></HostId></Error>" Duration: 47.083495ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............XX..............................................] duration: 476.353368ms FAIL (2025-07-25 10:13:27 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== oad=21019]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:40.996438590 (21h58m23.841240106s) [noise=true , address=77.90.31.13:4627 , piece_id=HBRLEALALTYTQHBRC3OM7VC2AN2D4GAXKF7M36TAF5HD7G6OTPBQ, seg_pos={0 0}, upload=21019]: Done uploading piece. err: <nil> Jul 25 09:33:40.996877017 (21h58m23.841678534s) [noise=true , address=77.90.31.17:4772 , piece_id=HZP632HLOH3CYRU3AZHCLZSOEX7BL4F22YGSO5NM6GY3YCIJ26PA, seg_pos={0 0}, upload=21019]: Done uploading piece. err: <nil> Jul 25 09:33:40.996900235 (21h58m23.841701752s) [seg_pos={0 0}, upload=21019]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jul 25 09:33:40.996968719 (21h58m23.841770241s) [upload=21019]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:33:40.996975079 (21h58m23.841776601s) [upload=21019]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:41.050252368 (21h58m23.895054131s) [upload=21019]: Done uploading segments... Jul 25 09:33:41.050275330 (21h58m23.895076853s) [upload=21019]: Upload finished. err: <nil> Jul 25 09:34:01.395455482 (21h58m44.240257015s) [upload=21028]: Starting upload Jul 25 09:34:01.395607427 (21h58m44.240408952s) [upload=21028]: Done starting upload Jul 25 09:34:01.395825566 (21h58m44.240627098s) [upload=21028]: Uploading segments... Jul 25 09:34:01.396495420 (21h58m44.241296955s) [upload=21028]: Got next segment. Inline: false Jul 25 09:34:01.396604095 (21h58m44.241405672s) [upload=21028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:34:01.396612168 (21h58m44.241413704s) [upload=21028]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:34:01.425826935 (21h58m44.270628471s) [seg_pos={0 0}, upload=21028]: Begin upload segment... Jul 25 09:34:01.426020430 (21h58m44.270821966s) [seg_pos={0 0}, upload=21028]: Done begin upload segment. Jul 25 09:34:01.426040728 (21h58m44.270842262s) [noise=true , address=104.36.82.4:4801 , piece_id=3NQQNM47LKMBO5GAQTMIWFTCGMD74JPMV4UPN7DTSVEKEZDZV5CA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.426061340 (21h58m44.2708629s) [noise=true , address=45.91.116.9:4263 , piece_id=VQVHJLCTXK5RMM2COSMGLP3UERPAFXL7CGCXROLSTIS4MIDTXGJA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.426601379 (21h58m44.271402918s) [noise=true , address=45.91.116.13:4155 , piece_id=V522D5F5OHME2REAEHIKBXKB4TA7FA24AOMZW5SMZV65UG4KW3XQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.426656014 (21h58m44.271457552s) [noise=true , address=77.90.31.13:4627 , piece_id=4AQ2PIDAK475LLLAG63ECCYHSZYWYVI52EJG7K5L7PA5BNBICA2A, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.427123147 (21h58m44.271924683s) [noise=true , address=45.91.116.7:4010 , piece_id=5OIKMU4MAE77EW635NMFJ7DOZNTCPC3QEYK52P5VITJJEZWFKEEA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.427565449 (21h58m44.272366988s) [noise=true , address=45.91.116.10:4083 , piece_id=D4PFM5EKLWNM5U3WVVHT5BVFXHPPNRFS64FEYLZORCR63FVQOEIQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.428074025 (21h58m44.272875567s) [noise=true , address=45.91.116.13:4154 , piece_id=OI6I6PDJKSYMDPFX4Q7AUWFRUXKMAZM7KI2X5QKTRHU3ZLYYCMVQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.428510546 (21h58m44.273312077s) [noise=true , address=77.90.31.16:4735 , piece_id=6ZY3FR55LO6O5E56LW6I5I5R7CHPC2MGS7NUJDD7UAXLOS3GXK4Q, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.428934503 (21h58m44.273736038s) [noise=true , address=45.91.116.6:4975 , piece_id=MI33ERQ7USAD2VE72XW5GV7G7HVUTDWTMUIXZQIBQQYOEHLR2DKQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.429488020 (21h58m44.274289548s) [noise=true , address=104.36.82.2:4730 , piece_id=VJNUI5MBLNWTMXZPXZNCFTSS4HBH7SBRXMQSMS36TIJOVLMA6X7Q, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.430016372 (21h58m44.274817913s) [noise=true , address=77.90.31.14:4661 , piece_id=AVBIPY5PRBUYUDVPMXCUFNUDQ6KPGDFMFW4HYZGUFLAKNC5ZLXYA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.430716875 (21h58m44.275518412s) [noise=true , address=104.36.82.12:4149 , piece_id=25LO6D4NKMKRLQQ4JNAXOLO5VAYANARND6RPNP5AW6J2WD5JCDXQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.431223998 (21h58m44.27602553s) [noise=true , address=77.90.31.19:4842 , piece_id=QWKQ455RQK2Q7IS3U5EXHAW3FX33XWJDTEV2GPDDAL5FJTEGTPGQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.431734863 (21h58m44.2765364s) [noise=true , address=104.36.82.14:4223 , piece_id=KG47KSNPFPH7PHYPLBC4GONV5XLYDG6M3HUMWZLK7G7PU6HPCXQQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.432299394 (21h58m44.277100922s) [noise=true , address=45.91.116.4:4937 , piece_id=GYYBC7JJCP2VOENJJAZRDAO6NMHVZXZEXA5M3APOATJCXDQ5XSNA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.432742241 (21h58m44.277543768s) [noise=true , address=77.90.31.15:4698 , piece_id=UKE2SEHPFX367ON5N2THRO5XN26KWOMQLP7RD36K2H5JTLXIRWOA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.541163047 (21h58m44.385964577s) [upload=21028]: Next returned nil segment Jul 25 09:34:01.541183693 (21h58m44.385985227s) [upload=21028]: Waiting for error group managing segments... Jul 25 09:34:01.555640486 (21h58m44.400441996s) [noise=true , address=45.91.116.6:4975 , piece_id=MI33ERQ7USAD2VE72XW5GV7G7HVUTDWTMUIXZQIBQQYOEHLR2DKQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.556377594 (21h58m44.401179119s) [noise=true , address=45.91.116.10:4083 , piece_id=D4PFM5EKLWNM5U3WVVHT5BVFXHPPNRFS64FEYLZORCR63FVQOEIQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.557556943 (21h58m44.402358454s) [noise=true , address=45.91.116.13:4155 , piece_id=V522D5F5OHME2REAEHIKBXKB4TA7FA24AOMZW5SMZV65UG4KW3XQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.558013379 (21h58m44.402814904s) [noise=true , address=77.90.31.13:4627 , piece_id=4AQ2PIDAK475LLLAG63ECCYHSZYWYVI52EJG7K5L7PA5BNBICA2A, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.558428945 (21h58m44.403230459s) [noise=true , address=45.91.116.9:4263 , piece_id=VQVHJLCTXK5RMM2COSMGLP3UERPAFXL7CGCXROLSTIS4MIDTXGJA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.558956623 (21h58m44.403758141s) [noise=true , address=104.36.82.2:4730 , piece_id=VJNUI5MBLNWTMXZPXZNCFTSS4HBH7SBRXMQSMS36TIJOVLMA6X7Q, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.559007198 (21h58m44.403808715s) [noise=true , address=45.91.116.13:4154 , piece_id=OI6I6PDJKSYMDPFX4Q7AUWFRUXKMAZM7KI2X5QKTRHU3ZLYYCMVQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.559590455 (21h58m44.404391965s) [noise=true , address=45.91.116.4:4937 , piece_id=GYYBC7JJCP2VOENJJAZRDAO6NMHVZXZEXA5M3APOATJCXDQ5XSNA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.559696005 (21h58m44.404497534s) [noise=true , address=77.90.31.14:4661 , piece_id=AVBIPY5PRBUYUDVPMXCUFNUDQ6KPGDFMFW4HYZGUFLAKNC5ZLXYA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.560471920 (21h58m44.405273433s) [noise=true , address=104.36.82.14:4223 , piece_id=KG47KSNPFPH7PHYPLBC4GONV5XLYDG6M3HUMWZLK7G7PU6HPCXQQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561137208 (21h58m44.405938736s) [noise=true , address=77.90.31.15:4698 , piece_id=UKE2SEHPFX367ON5N2THRO5XN26KWOMQLP7RD36K2H5JTLXIRWOA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561216161 (21h58m44.406017672s) [noise=true , address=77.90.31.19:4842 , piece_id=QWKQ455RQK2Q7IS3U5EXHAW3FX33XWJDTEV2GPDDAL5FJTEGTPGQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561662523 (21h58m44.40646404s) [noise=true , address=45.91.116.7:4010 , piece_id=5OIKMU4MAE77EW635NMFJ7DOZNTCPC3QEYK52P5VITJJEZWFKEEA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561675230 (21h58m44.406476756s) [seg_pos={0 0}, upload=21028]: Segment reached optimal threshold of 13 pieces. Jul 25 09:34:01.561800391 (21h58m44.406601905s) [noise=true , address=104.36.82.12:4149 , piece_id=25LO6D4NKMKRLQQ4JNAXOLO5VAYANARND6RPNP5AW6J2WD5JCDXQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:01.561994528 (21h58m44.406796058s) [noise=true , address=104.36.82.4:4801 , piece_id=3NQQNM47LKMBO5GAQTMIWFTCGMD74JPMV4UPN7DTSVEKEZDZV5CA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:01.562128894 (21h58m44.406930406s) [noise=true , address=77.90.31.16:4735 , piece_id=6ZY3FR55LO6O5E56LW6I5I5R7CHPC2MGS7NUJDD7UAXLOS3GXK4Q, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.562229276 (21h58m44.407030809s) [seg_pos={0 0}, upload=21028]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 25 09:34:01.562358118 (21h58m44.407159707s) [upload=21028]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:34:01.562365734 (21h58m44.407167258s) [upload=21028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:01.612443727 (21h58m44.457245259s) [upload=21028]: Done uploading segments... Jul 25 09:34:01.612489093 (21h58m44.457290622s) [upload=21028]: Upload finished. err: <nil> Duration: 476.385039ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.........X............X...X.................................] duration: 446.688113ms FAIL (2025-07-25 10:13:24 UTC) orbiter warning: uplink: invalid range 0:1024 (size:1024) Duration: 446.715301ms core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 561.978261ms FAIL (2025-07-25 10:13:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 562.004749ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........XX.................................................] duration: 1.736746465s FAIL (2025-07-25 10:13:22 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 1.73677761s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............XX.............................................] duration: 433.438541ms FAIL (2025-07-25 10:13:07 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== oad=21019]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:40.996438590 (21h58m23.841240106s) [noise=true , address=77.90.31.13:4627 , piece_id=HBRLEALALTYTQHBRC3OM7VC2AN2D4GAXKF7M36TAF5HD7G6OTPBQ, seg_pos={0 0}, upload=21019]: Done uploading piece. err: <nil> Jul 25 09:33:40.996877017 (21h58m23.841678534s) [noise=true , address=77.90.31.17:4772 , piece_id=HZP632HLOH3CYRU3AZHCLZSOEX7BL4F22YGSO5NM6GY3YCIJ26PA, seg_pos={0 0}, upload=21019]: Done uploading piece. err: <nil> Jul 25 09:33:40.996900235 (21h58m23.841701752s) [seg_pos={0 0}, upload=21019]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jul 25 09:33:40.996968719 (21h58m23.841770241s) [upload=21019]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:33:40.996975079 (21h58m23.841776601s) [upload=21019]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:41.050252368 (21h58m23.895054131s) [upload=21019]: Done uploading segments... Jul 25 09:33:41.050275330 (21h58m23.895076853s) [upload=21019]: Upload finished. err: <nil> Jul 25 09:34:01.395455482 (21h58m44.240257015s) [upload=21028]: Starting upload Jul 25 09:34:01.395607427 (21h58m44.240408952s) [upload=21028]: Done starting upload Jul 25 09:34:01.395825566 (21h58m44.240627098s) [upload=21028]: Uploading segments... Jul 25 09:34:01.396495420 (21h58m44.241296955s) [upload=21028]: Got next segment. Inline: false Jul 25 09:34:01.396604095 (21h58m44.241405672s) [upload=21028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:34:01.396612168 (21h58m44.241413704s) [upload=21028]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:34:01.425826935 (21h58m44.270628471s) [seg_pos={0 0}, upload=21028]: Begin upload segment... Jul 25 09:34:01.426020430 (21h58m44.270821966s) [seg_pos={0 0}, upload=21028]: Done begin upload segment. Jul 25 09:34:01.426040728 (21h58m44.270842262s) [noise=true , address=104.36.82.4:4801 , piece_id=3NQQNM47LKMBO5GAQTMIWFTCGMD74JPMV4UPN7DTSVEKEZDZV5CA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.426061340 (21h58m44.2708629s) [noise=true , address=45.91.116.9:4263 , piece_id=VQVHJLCTXK5RMM2COSMGLP3UERPAFXL7CGCXROLSTIS4MIDTXGJA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.426601379 (21h58m44.271402918s) [noise=true , address=45.91.116.13:4155 , piece_id=V522D5F5OHME2REAEHIKBXKB4TA7FA24AOMZW5SMZV65UG4KW3XQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.426656014 (21h58m44.271457552s) [noise=true , address=77.90.31.13:4627 , piece_id=4AQ2PIDAK475LLLAG63ECCYHSZYWYVI52EJG7K5L7PA5BNBICA2A, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.427123147 (21h58m44.271924683s) [noise=true , address=45.91.116.7:4010 , piece_id=5OIKMU4MAE77EW635NMFJ7DOZNTCPC3QEYK52P5VITJJEZWFKEEA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.427565449 (21h58m44.272366988s) [noise=true , address=45.91.116.10:4083 , piece_id=D4PFM5EKLWNM5U3WVVHT5BVFXHPPNRFS64FEYLZORCR63FVQOEIQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.428074025 (21h58m44.272875567s) [noise=true , address=45.91.116.13:4154 , piece_id=OI6I6PDJKSYMDPFX4Q7AUWFRUXKMAZM7KI2X5QKTRHU3ZLYYCMVQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.428510546 (21h58m44.273312077s) [noise=true , address=77.90.31.16:4735 , piece_id=6ZY3FR55LO6O5E56LW6I5I5R7CHPC2MGS7NUJDD7UAXLOS3GXK4Q, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.428934503 (21h58m44.273736038s) [noise=true , address=45.91.116.6:4975 , piece_id=MI33ERQ7USAD2VE72XW5GV7G7HVUTDWTMUIXZQIBQQYOEHLR2DKQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.429488020 (21h58m44.274289548s) [noise=true , address=104.36.82.2:4730 , piece_id=VJNUI5MBLNWTMXZPXZNCFTSS4HBH7SBRXMQSMS36TIJOVLMA6X7Q, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.430016372 (21h58m44.274817913s) [noise=true , address=77.90.31.14:4661 , piece_id=AVBIPY5PRBUYUDVPMXCUFNUDQ6KPGDFMFW4HYZGUFLAKNC5ZLXYA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.430716875 (21h58m44.275518412s) [noise=true , address=104.36.82.12:4149 , piece_id=25LO6D4NKMKRLQQ4JNAXOLO5VAYANARND6RPNP5AW6J2WD5JCDXQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.431223998 (21h58m44.27602553s) [noise=true , address=77.90.31.19:4842 , piece_id=QWKQ455RQK2Q7IS3U5EXHAW3FX33XWJDTEV2GPDDAL5FJTEGTPGQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.431734863 (21h58m44.2765364s) [noise=true , address=104.36.82.14:4223 , piece_id=KG47KSNPFPH7PHYPLBC4GONV5XLYDG6M3HUMWZLK7G7PU6HPCXQQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.432299394 (21h58m44.277100922s) [noise=true , address=45.91.116.4:4937 , piece_id=GYYBC7JJCP2VOENJJAZRDAO6NMHVZXZEXA5M3APOATJCXDQ5XSNA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.432742241 (21h58m44.277543768s) [noise=true , address=77.90.31.15:4698 , piece_id=UKE2SEHPFX367ON5N2THRO5XN26KWOMQLP7RD36K2H5JTLXIRWOA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.541163047 (21h58m44.385964577s) [upload=21028]: Next returned nil segment Jul 25 09:34:01.541183693 (21h58m44.385985227s) [upload=21028]: Waiting for error group managing segments... Jul 25 09:34:01.555640486 (21h58m44.400441996s) [noise=true , address=45.91.116.6:4975 , piece_id=MI33ERQ7USAD2VE72XW5GV7G7HVUTDWTMUIXZQIBQQYOEHLR2DKQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.556377594 (21h58m44.401179119s) [noise=true , address=45.91.116.10:4083 , piece_id=D4PFM5EKLWNM5U3WVVHT5BVFXHPPNRFS64FEYLZORCR63FVQOEIQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.557556943 (21h58m44.402358454s) [noise=true , address=45.91.116.13:4155 , piece_id=V522D5F5OHME2REAEHIKBXKB4TA7FA24AOMZW5SMZV65UG4KW3XQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.558013379 (21h58m44.402814904s) [noise=true , address=77.90.31.13:4627 , piece_id=4AQ2PIDAK475LLLAG63ECCYHSZYWYVI52EJG7K5L7PA5BNBICA2A, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.558428945 (21h58m44.403230459s) [noise=true , address=45.91.116.9:4263 , piece_id=VQVHJLCTXK5RMM2COSMGLP3UERPAFXL7CGCXROLSTIS4MIDTXGJA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.558956623 (21h58m44.403758141s) [noise=true , address=104.36.82.2:4730 , piece_id=VJNUI5MBLNWTMXZPXZNCFTSS4HBH7SBRXMQSMS36TIJOVLMA6X7Q, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.559007198 (21h58m44.403808715s) [noise=true , address=45.91.116.13:4154 , piece_id=OI6I6PDJKSYMDPFX4Q7AUWFRUXKMAZM7KI2X5QKTRHU3ZLYYCMVQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.559590455 (21h58m44.404391965s) [noise=true , address=45.91.116.4:4937 , piece_id=GYYBC7JJCP2VOENJJAZRDAO6NMHVZXZEXA5M3APOATJCXDQ5XSNA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.559696005 (21h58m44.404497534s) [noise=true , address=77.90.31.14:4661 , piece_id=AVBIPY5PRBUYUDVPMXCUFNUDQ6KPGDFMFW4HYZGUFLAKNC5ZLXYA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.560471920 (21h58m44.405273433s) [noise=true , address=104.36.82.14:4223 , piece_id=KG47KSNPFPH7PHYPLBC4GONV5XLYDG6M3HUMWZLK7G7PU6HPCXQQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561137208 (21h58m44.405938736s) [noise=true , address=77.90.31.15:4698 , piece_id=UKE2SEHPFX367ON5N2THRO5XN26KWOMQLP7RD36K2H5JTLXIRWOA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561216161 (21h58m44.406017672s) [noise=true , address=77.90.31.19:4842 , piece_id=QWKQ455RQK2Q7IS3U5EXHAW3FX33XWJDTEV2GPDDAL5FJTEGTPGQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561662523 (21h58m44.40646404s) [noise=true , address=45.91.116.7:4010 , piece_id=5OIKMU4MAE77EW635NMFJ7DOZNTCPC3QEYK52P5VITJJEZWFKEEA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561675230 (21h58m44.406476756s) [seg_pos={0 0}, upload=21028]: Segment reached optimal threshold of 13 pieces. Jul 25 09:34:01.561800391 (21h58m44.406601905s) [noise=true , address=104.36.82.12:4149 , piece_id=25LO6D4NKMKRLQQ4JNAXOLO5VAYANARND6RPNP5AW6J2WD5JCDXQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:01.561994528 (21h58m44.406796058s) [noise=true , address=104.36.82.4:4801 , piece_id=3NQQNM47LKMBO5GAQTMIWFTCGMD74JPMV4UPN7DTSVEKEZDZV5CA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:01.562128894 (21h58m44.406930406s) [noise=true , address=77.90.31.16:4735 , piece_id=6ZY3FR55LO6O5E56LW6I5I5R7CHPC2MGS7NUJDD7UAXLOS3GXK4Q, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.562229276 (21h58m44.407030809s) [seg_pos={0 0}, upload=21028]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 25 09:34:01.562358118 (21h58m44.407159707s) [upload=21028]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:34:01.562365734 (21h58m44.407167258s) [upload=21028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:01.612443727 (21h58m44.457245259s) [upload=21028]: Done uploading segments... Jul 25 09:34:01.612489093 (21h58m44.457290622s) [upload=21028]: Upload finished. err: <nil> Duration: 433.465179ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 292.725426ms FAIL (2025-07-25 10:13:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 403, RequestID: 185574E861CBB3B7, HostID: , api error AccessDenied: Access Denied. Duration: 292.750841ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [........XXX.................................................] duration: 939.434334ms FAIL (2025-07-25 10:13:04 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 939.467813ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........XX................................................] duration: 1.847938761s FAIL (2025-07-25 10:13:01 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 1.847974986s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..........X............X...X................................] duration: 250.964105ms FAIL (2025-07-25 10:13:01 UTC) orbiter warning: uplink: invalid range 0:1024 (size:1024) Duration: 251.008238ms core uplink us1 vivint
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 82.185258ms remote: "tcp:45.38.24.2:443" response headers: Content-Length=279 Content-Security-Policy=block-all-mixed-content X-Request-Id=D5AmxJ2hsti X-Xss-Protection=1; mode=block Date=Fri, 25 Jul 2025 09:36:06 GMT Accept-Ranges=bytes Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=185574E451263CD8 target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-07-25 10:12:58 UTC) orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Access Denied.</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185574E451263CD8</RequestId><HostId></HostId></Error>" Duration: 82.208158ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 894.272803ms FAIL (2025-07-25 10:12:46 UTC) orbiter warning: ======ERROR====== uplink: permission denied (bucket: metaclient: Unauthorized API credentials) ======LOGS====== h item: pb.BatchRequestItem_SegmentBegin Jul 25 09:35:15.918314780 (21h59m58.763116293s) [seg_pos={0 0}, upload=21044]: Begin upload segment... Jul 25 09:35:15.918358070 (21h59m58.763159586s) [seg_pos={0 0}, upload=21044]: Done begin upload segment. Jul 25 09:35:15.918382251 (21h59m58.763183776s) [noise=true , address=77.90.31.14:4678 , piece_id=HIZS7J3DY6Q33PO2AI6UNXQ5S54L6DGG6ZAP3YKZZSEUINZCXEPQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.918407152 (21h59m58.763208685s) [noise=true , address=45.91.116.6:4994 , piece_id=JPWTQR5RKBZACY6XJDNQAEUCFDNYFLC7BYZCLHOZCCFJPXEKOSMA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.918842345 (21h59m58.763643878s) [noise=true , address=77.90.31.16:4751 , piece_id=DCODR5LGROHG7DVIUSRLFRRBKR672JGYQ3JOOTJSLL2NVLY6L62Q, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.918965591 (21h59m58.763767123s) [noise=true , address=77.90.31.15:4720 , piece_id=FL7PFV4B2TZ54WTT3YYU3NPGT7RBNGGHRUS4QTOARDZASB7PKAGQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.919180183 (21h59m58.763981708s) [noise=true , address=77.90.31.18:4830 , piece_id=2Z6ELONZADYWJJZPQUWKPTARWQI5D4CRE7XDIQBCUVVWAOUOYFGA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.919362724 (21h59m58.76416425s) [noise=true , address=79.127.205.235:4690 , piece_id=4MLADNII3WTUAMY5KKWUM2XSVFDR3NSLA32EMVJ3AIDOJENSF3SA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.919457169 (21h59m58.764258696s) [noise=true , address=79.127.205.246:4263 , piece_id=2RB4RONQXXPKZIETGCVEOM2JQUC5EIKETQA6VKCSSHF6ARLG2BRQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.919677786 (21h59m58.764479318s) [noise=true , address=104.36.82.15:4284 , piece_id=3OMO32MXTLOB6JUNQQVR27UNL5CHYN5XWWG3TO3DUOOIMEWYYY3A, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.919849288 (21h59m58.764650816s) [noise=true , address=45.91.116.10:4099 , piece_id=CWLXVFK56DYPPB3ED5INVEHKBUPW6MSKZLAA7SAFQU3YFT3UNHXQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.920171246 (21h59m58.764972769s) [noise=true , address=104.36.82.9:4021 , piece_id=T3KBEAMBTCD4I6UO52RTLVDFS47BRXJCZMGU44VEQITYQPM4K2YA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.920199227 (21h59m58.765000758s) [noise=true , address=104.36.82.50:4046 , piece_id=6UXAZY6MUTWLLVYZJBV4AYYIYH3KDR2IT22BWGGKBYJQGGMZGVIA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.920446438 (21h59m58.765247963s) [noise=true , address=77.90.31.19:4850 , piece_id=5FE4KIHO7B5CQ2ZHGB6CIZ4BA4ZYHMG666JB7S7YI3ZV25PM7DRQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.920500425 (21h59m58.765301948s) [noise=true , address=79.127.220.97:4367 , piece_id=466Y2CJB2PZ6EYNZLJAGIIXJHNAWS5P6L2GJXDWTYGA5CPVG2L3A, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.920773391 (21h59m58.765574929s) [noise=true , address=104.36.82.46:4509 , piece_id=LF6MNWECHJUQIIGTCYSQAMVWOUVYAJ7S7WPPANUYNDVVZPD26QCA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.920795516 (21h59m58.765597039s) [noise=true , address=79.127.219.47:4065 , piece_id=63NEETIH55O2QLL6OTFXXIGRE4MSCVSOSUKPNTCNHYENBJKPA5TQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.921069667 (21h59m58.76587119s) [noise=true , address=77.90.31.17:4790 , piece_id=PQGH3COTKNFS37FWGA6FLRVASZTELDOMRZVY5DGHTMVGZZIQMQWQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.921112088 (21h59m58.765913613s) [noise=true , address=79.127.201.215:4350 , piece_id=5V225ATIAYECLX2LXE2H5RZFJJCEAVTKIFCGKYWBOF532LOMGJJQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.921402936 (21h59m58.766204463s) [noise=true , address=45.91.116.7:4040 , piece_id=GFCCEQ5NJD4J7QZAXY7AEHZMCCRV3NHLNTK76YWW2TMOK46GKJQA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:16.026469477 (21h59m58.871270992s) [upload=21044]: Next returned nil segment Jul 25 09:35:16.026486428 (21h59m58.871287959s) [upload=21044]: Waiting for error group managing segments... Jul 25 09:35:16.032614751 (21h59m58.877416267s) [noise=true , address=77.90.31.19:4850 , piece_id=5FE4KIHO7B5CQ2ZHGB6CIZ4BA4ZYHMG666JB7S7YI3ZV25PM7DRQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.033106703 (21h59m58.877908219s) [noise=true , address=45.91.116.6:4994 , piece_id=JPWTQR5RKBZACY6XJDNQAEUCFDNYFLC7BYZCLHOZCCFJPXEKOSMA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.034162031 (21h59m58.87896354s) [noise=true , address=77.90.31.18:4830 , piece_id=2Z6ELONZADYWJJZPQUWKPTARWQI5D4CRE7XDIQBCUVVWAOUOYFGA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.036556969 (21h59m58.881358495s) [noise=true , address=104.36.82.46:4509 , piece_id=LF6MNWECHJUQIIGTCYSQAMVWOUVYAJ7S7WPPANUYNDVVZPD26QCA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.036598532 (21h59m58.881400048s) [noise=true , address=104.36.82.15:4284 , piece_id=3OMO32MXTLOB6JUNQQVR27UNL5CHYN5XWWG3TO3DUOOIMEWYYY3A, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.042836507 (21h59m58.887638032s) [noise=true , address=79.127.219.47:4065 , piece_id=63NEETIH55O2QLL6OTFXXIGRE4MSCVSOSUKPNTCNHYENBJKPA5TQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.046300094 (21h59m58.891101617s) [noise=true , address=77.90.31.14:4678 , piece_id=HIZS7J3DY6Q33PO2AI6UNXQ5S54L6DGG6ZAP3YKZZSEUINZCXEPQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.046770857 (21h59m58.891572381s) [noise=true , address=77.90.31.17:4790 , piece_id=PQGH3COTKNFS37FWGA6FLRVASZTELDOMRZVY5DGHTMVGZZIQMQWQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.047620050 (21h59m58.892421564s) [noise=true , address=104.36.82.9:4021 , piece_id=T3KBEAMBTCD4I6UO52RTLVDFS47BRXJCZMGU44VEQITYQPM4K2YA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.049041308 (21h59m58.893842842s) [noise=true , address=45.91.116.7:4040 , piece_id=GFCCEQ5NJD4J7QZAXY7AEHZMCCRV3NHLNTK76YWW2TMOK46GKJQA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.049699949 (21h59m58.894501458s) [noise=true , address=45.91.116.10:4099 , piece_id=CWLXVFK56DYPPB3ED5INVEHKBUPW6MSKZLAA7SAFQU3YFT3UNHXQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.058570793 (21h59m58.903372309s) [noise=true , address=104.36.82.50:4046 , piece_id=6UXAZY6MUTWLLVYZJBV4AYYIYH3KDR2IT22BWGGKBYJQGGMZGVIA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.063456832 (21h59m58.908258348s) [noise=true , address=77.90.31.16:4751 , piece_id=DCODR5LGROHG7DVIUSRLFRRBKR672JGYQ3JOOTJSLL2NVLY6L62Q, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.064332620 (21h59m58.909134133s) [noise=true , address=77.90.31.15:4720 , piece_id=FL7PFV4B2TZ54WTT3YYU3NPGT7RBNGGHRUS4QTOARDZASB7PKAGQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.064383494 (21h59m58.909185033s) [seg_pos={0 0}, upload=21044]: Segment reached optimal threshold of 14 pieces. Jul 25 09:35:16.064635168 (21h59m58.909436691s) [noise=true , address=79.127.220.97:4367 , piece_id=466Y2CJB2PZ6EYNZLJAGIIXJHNAWS5P6L2GJXDWTYGA5CPVG2L3A, seg_pos={0 0}, upload=21044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ukTBGQCf81aFiZhST9sqJEN28d1qg44gv27U2oPJKiGNirRPB): protocol: write tcp 159.223.143.39:49600->79.127.220.97:4367: use of closed network connection; write tcp 159.223.143.39:49600->79.127.220.97:4367: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:49600->79.127.220.97:4367: use of closed network connection Jul 25 09:35:16.064733953 (21h59m58.909535466s) [noise=true , address=79.127.205.235:4690 , piece_id=4MLADNII3WTUAMY5KKWUM2XSVFDR3NSLA32EMVJ3AIDOJENSF3SA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121SG1z7mpp74ipZ9b7hVE9WrEicBpRk84uoc3JW7FKupxKLcdg): protocol: read tcp 159.223.143.39:49260->79.127.205.235:4690: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 25 09:35:16.064916028 (21h59m58.90971754s) [noise=true , address=79.127.201.215:4350 , piece_id=5V225ATIAYECLX2LXE2H5RZFJJCEAVTKIFCGKYWBOF532LOMGJJQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WhZAWvHCTj5xVkUuVZtgkFB1dhFKpBPq18YMKkepFx5qxkUAr): protocol: read tcp 159.223.143.39:39812->79.127.201.215:4350: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 25 09:35:16.064952221 (21h59m58.909753734s) [noise=true , address=79.127.205.246:4263 , piece_id=2RB4RONQXXPKZIETGCVEOM2JQUC5EIKETQA6VKCSSHF6ARLG2BRQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZV35gnSB9uZZZQDr9v8esb7r4XH6TcxLdnC8LnESkoKxH2FZt): protocol: read tcp 159.223.143.39:54542->79.127.205.246:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 25 09:35:16.064963992 (21h59m58.909765516s) [seg_pos={0 0}, upload=21044]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 25 09:35:16.065052663 (21h59m58.909854191s) [upload=21044]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:35:16.065058796 (21h59m58.909860327s) [upload=21044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:35:16.170721369 (21h59m59.015522894s) [upload=21044]: Done uploading segments... Jul 25 09:35:16.170744580 (21h59m59.015546107s) [upload=21044]: Upload finished. err: <nil> Duration:* 894.299072ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............XX............................................] duration: 527.752323ms FAIL (2025-07-25 10:12:44 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== oad=21019]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:40.996438590 (21h58m23.841240106s) [noise=true , address=77.90.31.13:4627 , piece_id=HBRLEALALTYTQHBRC3OM7VC2AN2D4GAXKF7M36TAF5HD7G6OTPBQ, seg_pos={0 0}, upload=21019]: Done uploading piece. err: <nil> Jul 25 09:33:40.996877017 (21h58m23.841678534s) [noise=true , address=77.90.31.17:4772 , piece_id=HZP632HLOH3CYRU3AZHCLZSOEX7BL4F22YGSO5NM6GY3YCIJ26PA, seg_pos={0 0}, upload=21019]: Done uploading piece. err: <nil> Jul 25 09:33:40.996900235 (21h58m23.841701752s) [seg_pos={0 0}, upload=21019]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jul 25 09:33:40.996968719 (21h58m23.841770241s) [upload=21019]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:33:40.996975079 (21h58m23.841776601s) [upload=21019]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:41.050252368 (21h58m23.895054131s) [upload=21019]: Done uploading segments... Jul 25 09:33:41.050275330 (21h58m23.895076853s) [upload=21019]: Upload finished. err: <nil> Jul 25 09:34:01.395455482 (21h58m44.240257015s) [upload=21028]: Starting upload Jul 25 09:34:01.395607427 (21h58m44.240408952s) [upload=21028]: Done starting upload Jul 25 09:34:01.395825566 (21h58m44.240627098s) [upload=21028]: Uploading segments... Jul 25 09:34:01.396495420 (21h58m44.241296955s) [upload=21028]: Got next segment. Inline: false Jul 25 09:34:01.396604095 (21h58m44.241405672s) [upload=21028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:34:01.396612168 (21h58m44.241413704s) [upload=21028]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:34:01.425826935 (21h58m44.270628471s) [seg_pos={0 0}, upload=21028]: Begin upload segment... Jul 25 09:34:01.426020430 (21h58m44.270821966s) [seg_pos={0 0}, upload=21028]: Done begin upload segment. Jul 25 09:34:01.426040728 (21h58m44.270842262s) [noise=true , address=104.36.82.4:4801 , piece_id=3NQQNM47LKMBO5GAQTMIWFTCGMD74JPMV4UPN7DTSVEKEZDZV5CA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.426061340 (21h58m44.2708629s) [noise=true , address=45.91.116.9:4263 , piece_id=VQVHJLCTXK5RMM2COSMGLP3UERPAFXL7CGCXROLSTIS4MIDTXGJA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.426601379 (21h58m44.271402918s) [noise=true , address=45.91.116.13:4155 , piece_id=V522D5F5OHME2REAEHIKBXKB4TA7FA24AOMZW5SMZV65UG4KW3XQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.426656014 (21h58m44.271457552s) [noise=true , address=77.90.31.13:4627 , piece_id=4AQ2PIDAK475LLLAG63ECCYHSZYWYVI52EJG7K5L7PA5BNBICA2A, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.427123147 (21h58m44.271924683s) [noise=true , address=45.91.116.7:4010 , piece_id=5OIKMU4MAE77EW635NMFJ7DOZNTCPC3QEYK52P5VITJJEZWFKEEA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.427565449 (21h58m44.272366988s) [noise=true , address=45.91.116.10:4083 , piece_id=D4PFM5EKLWNM5U3WVVHT5BVFXHPPNRFS64FEYLZORCR63FVQOEIQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.428074025 (21h58m44.272875567s) [noise=true , address=45.91.116.13:4154 , piece_id=OI6I6PDJKSYMDPFX4Q7AUWFRUXKMAZM7KI2X5QKTRHU3ZLYYCMVQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.428510546 (21h58m44.273312077s) [noise=true , address=77.90.31.16:4735 , piece_id=6ZY3FR55LO6O5E56LW6I5I5R7CHPC2MGS7NUJDD7UAXLOS3GXK4Q, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.428934503 (21h58m44.273736038s) [noise=true , address=45.91.116.6:4975 , piece_id=MI33ERQ7USAD2VE72XW5GV7G7HVUTDWTMUIXZQIBQQYOEHLR2DKQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.429488020 (21h58m44.274289548s) [noise=true , address=104.36.82.2:4730 , piece_id=VJNUI5MBLNWTMXZPXZNCFTSS4HBH7SBRXMQSMS36TIJOVLMA6X7Q, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.430016372 (21h58m44.274817913s) [noise=true , address=77.90.31.14:4661 , piece_id=AVBIPY5PRBUYUDVPMXCUFNUDQ6KPGDFMFW4HYZGUFLAKNC5ZLXYA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.430716875 (21h58m44.275518412s) [noise=true , address=104.36.82.12:4149 , piece_id=25LO6D4NKMKRLQQ4JNAXOLO5VAYANARND6RPNP5AW6J2WD5JCDXQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.431223998 (21h58m44.27602553s) [noise=true , address=77.90.31.19:4842 , piece_id=QWKQ455RQK2Q7IS3U5EXHAW3FX33XWJDTEV2GPDDAL5FJTEGTPGQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.431734863 (21h58m44.2765364s) [noise=true , address=104.36.82.14:4223 , piece_id=KG47KSNPFPH7PHYPLBC4GONV5XLYDG6M3HUMWZLK7G7PU6HPCXQQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.432299394 (21h58m44.277100922s) [noise=true , address=45.91.116.4:4937 , piece_id=GYYBC7JJCP2VOENJJAZRDAO6NMHVZXZEXA5M3APOATJCXDQ5XSNA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.432742241 (21h58m44.277543768s) [noise=true , address=77.90.31.15:4698 , piece_id=UKE2SEHPFX367ON5N2THRO5XN26KWOMQLP7RD36K2H5JTLXIRWOA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.541163047 (21h58m44.385964577s) [upload=21028]: Next returned nil segment Jul 25 09:34:01.541183693 (21h58m44.385985227s) [upload=21028]: Waiting for error group managing segments... Jul 25 09:34:01.555640486 (21h58m44.400441996s) [noise=true , address=45.91.116.6:4975 , piece_id=MI33ERQ7USAD2VE72XW5GV7G7HVUTDWTMUIXZQIBQQYOEHLR2DKQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.556377594 (21h58m44.401179119s) [noise=true , address=45.91.116.10:4083 , piece_id=D4PFM5EKLWNM5U3WVVHT5BVFXHPPNRFS64FEYLZORCR63FVQOEIQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.557556943 (21h58m44.402358454s) [noise=true , address=45.91.116.13:4155 , piece_id=V522D5F5OHME2REAEHIKBXKB4TA7FA24AOMZW5SMZV65UG4KW3XQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.558013379 (21h58m44.402814904s) [noise=true , address=77.90.31.13:4627 , piece_id=4AQ2PIDAK475LLLAG63ECCYHSZYWYVI52EJG7K5L7PA5BNBICA2A, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.558428945 (21h58m44.403230459s) [noise=true , address=45.91.116.9:4263 , piece_id=VQVHJLCTXK5RMM2COSMGLP3UERPAFXL7CGCXROLSTIS4MIDTXGJA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.558956623 (21h58m44.403758141s) [noise=true , address=104.36.82.2:4730 , piece_id=VJNUI5MBLNWTMXZPXZNCFTSS4HBH7SBRXMQSMS36TIJOVLMA6X7Q, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.559007198 (21h58m44.403808715s) [noise=true , address=45.91.116.13:4154 , piece_id=OI6I6PDJKSYMDPFX4Q7AUWFRUXKMAZM7KI2X5QKTRHU3ZLYYCMVQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.559590455 (21h58m44.404391965s) [noise=true , address=45.91.116.4:4937 , piece_id=GYYBC7JJCP2VOENJJAZRDAO6NMHVZXZEXA5M3APOATJCXDQ5XSNA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.559696005 (21h58m44.404497534s) [noise=true , address=77.90.31.14:4661 , piece_id=AVBIPY5PRBUYUDVPMXCUFNUDQ6KPGDFMFW4HYZGUFLAKNC5ZLXYA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.560471920 (21h58m44.405273433s) [noise=true , address=104.36.82.14:4223 , piece_id=KG47KSNPFPH7PHYPLBC4GONV5XLYDG6M3HUMWZLK7G7PU6HPCXQQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561137208 (21h58m44.405938736s) [noise=true , address=77.90.31.15:4698 , piece_id=UKE2SEHPFX367ON5N2THRO5XN26KWOMQLP7RD36K2H5JTLXIRWOA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561216161 (21h58m44.406017672s) [noise=true , address=77.90.31.19:4842 , piece_id=QWKQ455RQK2Q7IS3U5EXHAW3FX33XWJDTEV2GPDDAL5FJTEGTPGQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561662523 (21h58m44.40646404s) [noise=true , address=45.91.116.7:4010 , piece_id=5OIKMU4MAE77EW635NMFJ7DOZNTCPC3QEYK52P5VITJJEZWFKEEA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561675230 (21h58m44.406476756s) [seg_pos={0 0}, upload=21028]: Segment reached optimal threshold of 13 pieces. Jul 25 09:34:01.561800391 (21h58m44.406601905s) [noise=true , address=104.36.82.12:4149 , piece_id=25LO6D4NKMKRLQQ4JNAXOLO5VAYANARND6RPNP5AW6J2WD5JCDXQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:01.561994528 (21h58m44.406796058s) [noise=true , address=104.36.82.4:4801 , piece_id=3NQQNM47LKMBO5GAQTMIWFTCGMD74JPMV4UPN7DTSVEKEZDZV5CA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:01.562128894 (21h58m44.406930406s) [noise=true , address=77.90.31.16:4735 , piece_id=6ZY3FR55LO6O5E56LW6I5I5R7CHPC2MGS7NUJDD7UAXLOS3GXK4Q, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.562229276 (21h58m44.407030809s) [seg_pos={0 0}, upload=21028]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 25 09:34:01.562358118 (21h58m44.407159707s) [upload=21028]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:34:01.562365734 (21h58m44.407167258s) [upload=21028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:01.612443727 (21h58m44.457245259s) [upload=21028]: Done uploading segments... Jul 25 09:34:01.612489093 (21h58m44.457290622s) [upload=21028]: Upload finished. err: <nil> Duration: 527.77651ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 369.125294ms FAIL (2025-07-25 10:12:44 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 369.153593ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...........X............X...X...............................] duration: 242.401225ms FAIL (2025-07-25 10:12:32 UTC) orbiter warning: uplink: invalid range 0:1024 (size:1024) Duration: 242.429217ms core uplink us1 vivint
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 49.87552ms remote: "tcp:45.38.24.2:443" response headers: Content-Length=279 Content-Security-Policy=block-all-mixed-content X-Request-Id=D5AmxJ2hsti X-Xss-Protection=1; mode=block Date=Fri, 25 Jul 2025 09:36:06 GMT Accept-Ranges=bytes Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=185574E451263CD8 target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-07-25 10:12:29 UTC) orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Access Denied.</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185574E451263CD8</RequestId><HostId></HostId></Error>" Duration: 49.901517ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 300.424768ms FAIL (2025-07-25 10:12:24 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 300.474538ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............XX...........................................] duration: 495.056191ms FAIL (2025-07-25 10:12:22 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== oad=21019]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:40.996438590 (21h58m23.841240106s) [noise=true , address=77.90.31.13:4627 , piece_id=HBRLEALALTYTQHBRC3OM7VC2AN2D4GAXKF7M36TAF5HD7G6OTPBQ, seg_pos={0 0}, upload=21019]: Done uploading piece. err: <nil> Jul 25 09:33:40.996877017 (21h58m23.841678534s) [noise=true , address=77.90.31.17:4772 , piece_id=HZP632HLOH3CYRU3AZHCLZSOEX7BL4F22YGSO5NM6GY3YCIJ26PA, seg_pos={0 0}, upload=21019]: Done uploading piece. err: <nil> Jul 25 09:33:40.996900235 (21h58m23.841701752s) [seg_pos={0 0}, upload=21019]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jul 25 09:33:40.996968719 (21h58m23.841770241s) [upload=21019]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:33:40.996975079 (21h58m23.841776601s) [upload=21019]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:41.050252368 (21h58m23.895054131s) [upload=21019]: Done uploading segments... Jul 25 09:33:41.050275330 (21h58m23.895076853s) [upload=21019]: Upload finished. err: <nil> Jul 25 09:34:01.395455482 (21h58m44.240257015s) [upload=21028]: Starting upload Jul 25 09:34:01.395607427 (21h58m44.240408952s) [upload=21028]: Done starting upload Jul 25 09:34:01.395825566 (21h58m44.240627098s) [upload=21028]: Uploading segments... Jul 25 09:34:01.396495420 (21h58m44.241296955s) [upload=21028]: Got next segment. Inline: false Jul 25 09:34:01.396604095 (21h58m44.241405672s) [upload=21028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:34:01.396612168 (21h58m44.241413704s) [upload=21028]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:34:01.425826935 (21h58m44.270628471s) [seg_pos={0 0}, upload=21028]: Begin upload segment... Jul 25 09:34:01.426020430 (21h58m44.270821966s) [seg_pos={0 0}, upload=21028]: Done begin upload segment. Jul 25 09:34:01.426040728 (21h58m44.270842262s) [noise=true , address=104.36.82.4:4801 , piece_id=3NQQNM47LKMBO5GAQTMIWFTCGMD74JPMV4UPN7DTSVEKEZDZV5CA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.426061340 (21h58m44.2708629s) [noise=true , address=45.91.116.9:4263 , piece_id=VQVHJLCTXK5RMM2COSMGLP3UERPAFXL7CGCXROLSTIS4MIDTXGJA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.426601379 (21h58m44.271402918s) [noise=true , address=45.91.116.13:4155 , piece_id=V522D5F5OHME2REAEHIKBXKB4TA7FA24AOMZW5SMZV65UG4KW3XQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.426656014 (21h58m44.271457552s) [noise=true , address=77.90.31.13:4627 , piece_id=4AQ2PIDAK475LLLAG63ECCYHSZYWYVI52EJG7K5L7PA5BNBICA2A, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.427123147 (21h58m44.271924683s) [noise=true , address=45.91.116.7:4010 , piece_id=5OIKMU4MAE77EW635NMFJ7DOZNTCPC3QEYK52P5VITJJEZWFKEEA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.427565449 (21h58m44.272366988s) [noise=true , address=45.91.116.10:4083 , piece_id=D4PFM5EKLWNM5U3WVVHT5BVFXHPPNRFS64FEYLZORCR63FVQOEIQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.428074025 (21h58m44.272875567s) [noise=true , address=45.91.116.13:4154 , piece_id=OI6I6PDJKSYMDPFX4Q7AUWFRUXKMAZM7KI2X5QKTRHU3ZLYYCMVQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.428510546 (21h58m44.273312077s) [noise=true , address=77.90.31.16:4735 , piece_id=6ZY3FR55LO6O5E56LW6I5I5R7CHPC2MGS7NUJDD7UAXLOS3GXK4Q, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.428934503 (21h58m44.273736038s) [noise=true , address=45.91.116.6:4975 , piece_id=MI33ERQ7USAD2VE72XW5GV7G7HVUTDWTMUIXZQIBQQYOEHLR2DKQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.429488020 (21h58m44.274289548s) [noise=true , address=104.36.82.2:4730 , piece_id=VJNUI5MBLNWTMXZPXZNCFTSS4HBH7SBRXMQSMS36TIJOVLMA6X7Q, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.430016372 (21h58m44.274817913s) [noise=true , address=77.90.31.14:4661 , piece_id=AVBIPY5PRBUYUDVPMXCUFNUDQ6KPGDFMFW4HYZGUFLAKNC5ZLXYA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.430716875 (21h58m44.275518412s) [noise=true , address=104.36.82.12:4149 , piece_id=25LO6D4NKMKRLQQ4JNAXOLO5VAYANARND6RPNP5AW6J2WD5JCDXQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.431223998 (21h58m44.27602553s) [noise=true , address=77.90.31.19:4842 , piece_id=QWKQ455RQK2Q7IS3U5EXHAW3FX33XWJDTEV2GPDDAL5FJTEGTPGQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.431734863 (21h58m44.2765364s) [noise=true , address=104.36.82.14:4223 , piece_id=KG47KSNPFPH7PHYPLBC4GONV5XLYDG6M3HUMWZLK7G7PU6HPCXQQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.432299394 (21h58m44.277100922s) [noise=true , address=45.91.116.4:4937 , piece_id=GYYBC7JJCP2VOENJJAZRDAO6NMHVZXZEXA5M3APOATJCXDQ5XSNA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.432742241 (21h58m44.277543768s) [noise=true , address=77.90.31.15:4698 , piece_id=UKE2SEHPFX367ON5N2THRO5XN26KWOMQLP7RD36K2H5JTLXIRWOA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.541163047 (21h58m44.385964577s) [upload=21028]: Next returned nil segment Jul 25 09:34:01.541183693 (21h58m44.385985227s) [upload=21028]: Waiting for error group managing segments... Jul 25 09:34:01.555640486 (21h58m44.400441996s) [noise=true , address=45.91.116.6:4975 , piece_id=MI33ERQ7USAD2VE72XW5GV7G7HVUTDWTMUIXZQIBQQYOEHLR2DKQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.556377594 (21h58m44.401179119s) [noise=true , address=45.91.116.10:4083 , piece_id=D4PFM5EKLWNM5U3WVVHT5BVFXHPPNRFS64FEYLZORCR63FVQOEIQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.557556943 (21h58m44.402358454s) [noise=true , address=45.91.116.13:4155 , piece_id=V522D5F5OHME2REAEHIKBXKB4TA7FA24AOMZW5SMZV65UG4KW3XQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.558013379 (21h58m44.402814904s) [noise=true , address=77.90.31.13:4627 , piece_id=4AQ2PIDAK475LLLAG63ECCYHSZYWYVI52EJG7K5L7PA5BNBICA2A, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.558428945 (21h58m44.403230459s) [noise=true , address=45.91.116.9:4263 , piece_id=VQVHJLCTXK5RMM2COSMGLP3UERPAFXL7CGCXROLSTIS4MIDTXGJA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.558956623 (21h58m44.403758141s) [noise=true , address=104.36.82.2:4730 , piece_id=VJNUI5MBLNWTMXZPXZNCFTSS4HBH7SBRXMQSMS36TIJOVLMA6X7Q, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.559007198 (21h58m44.403808715s) [noise=true , address=45.91.116.13:4154 , piece_id=OI6I6PDJKSYMDPFX4Q7AUWFRUXKMAZM7KI2X5QKTRHU3ZLYYCMVQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.559590455 (21h58m44.404391965s) [noise=true , address=45.91.116.4:4937 , piece_id=GYYBC7JJCP2VOENJJAZRDAO6NMHVZXZEXA5M3APOATJCXDQ5XSNA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.559696005 (21h58m44.404497534s) [noise=true , address=77.90.31.14:4661 , piece_id=AVBIPY5PRBUYUDVPMXCUFNUDQ6KPGDFMFW4HYZGUFLAKNC5ZLXYA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.560471920 (21h58m44.405273433s) [noise=true , address=104.36.82.14:4223 , piece_id=KG47KSNPFPH7PHYPLBC4GONV5XLYDG6M3HUMWZLK7G7PU6HPCXQQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561137208 (21h58m44.405938736s) [noise=true , address=77.90.31.15:4698 , piece_id=UKE2SEHPFX367ON5N2THRO5XN26KWOMQLP7RD36K2H5JTLXIRWOA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561216161 (21h58m44.406017672s) [noise=true , address=77.90.31.19:4842 , piece_id=QWKQ455RQK2Q7IS3U5EXHAW3FX33XWJDTEV2GPDDAL5FJTEGTPGQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561662523 (21h58m44.40646404s) [noise=true , address=45.91.116.7:4010 , piece_id=5OIKMU4MAE77EW635NMFJ7DOZNTCPC3QEYK52P5VITJJEZWFKEEA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561675230 (21h58m44.406476756s) [seg_pos={0 0}, upload=21028]: Segment reached optimal threshold of 13 pieces. Jul 25 09:34:01.561800391 (21h58m44.406601905s) [noise=true , address=104.36.82.12:4149 , piece_id=25LO6D4NKMKRLQQ4JNAXOLO5VAYANARND6RPNP5AW6J2WD5JCDXQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:01.561994528 (21h58m44.406796058s) [noise=true , address=104.36.82.4:4801 , piece_id=3NQQNM47LKMBO5GAQTMIWFTCGMD74JPMV4UPN7DTSVEKEZDZV5CA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:01.562128894 (21h58m44.406930406s) [noise=true , address=77.90.31.16:4735 , piece_id=6ZY3FR55LO6O5E56LW6I5I5R7CHPC2MGS7NUJDD7UAXLOS3GXK4Q, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.562229276 (21h58m44.407030809s) [seg_pos={0 0}, upload=21028]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 25 09:34:01.562358118 (21h58m44.407159707s) [upload=21028]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:34:01.562365734 (21h58m44.407167258s) [upload=21028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:01.612443727 (21h58m44.457245259s) [upload=21028]: Done uploading segments... Jul 25 09:34:01.612489093 (21h58m44.457290622s) [upload=21028]: Upload finished. err: <nil> Duration: 495.086678ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 520.430401ms FAIL (2025-07-25 10:12:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 520.459871ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........XX...............................................] duration: 1.827980587s FAIL (2025-07-25 10:12:14 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 1.828015809s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 321.629121ms FAIL (2025-07-25 10:12:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 403, RequestID: 185574E861CBB3B7, HostID: , api error AccessDenied: Access Denied. Duration: 321.654382ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........XXX................................................] duration: 509.04315ms FAIL (2025-07-25 10:12:05 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 509.068039ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................XX..........................................] duration: 461.843494ms FAIL (2025-07-25 10:11:58 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== oad=21019]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:40.996438590 (21h58m23.841240106s) [noise=true , address=77.90.31.13:4627 , piece_id=HBRLEALALTYTQHBRC3OM7VC2AN2D4GAXKF7M36TAF5HD7G6OTPBQ, seg_pos={0 0}, upload=21019]: Done uploading piece. err: <nil> Jul 25 09:33:40.996877017 (21h58m23.841678534s) [noise=true , address=77.90.31.17:4772 , piece_id=HZP632HLOH3CYRU3AZHCLZSOEX7BL4F22YGSO5NM6GY3YCIJ26PA, seg_pos={0 0}, upload=21019]: Done uploading piece. err: <nil> Jul 25 09:33:40.996900235 (21h58m23.841701752s) [seg_pos={0 0}, upload=21019]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jul 25 09:33:40.996968719 (21h58m23.841770241s) [upload=21019]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:33:40.996975079 (21h58m23.841776601s) [upload=21019]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:41.050252368 (21h58m23.895054131s) [upload=21019]: Done uploading segments... Jul 25 09:33:41.050275330 (21h58m23.895076853s) [upload=21019]: Upload finished. err: <nil> Jul 25 09:34:01.395455482 (21h58m44.240257015s) [upload=21028]: Starting upload Jul 25 09:34:01.395607427 (21h58m44.240408952s) [upload=21028]: Done starting upload Jul 25 09:34:01.395825566 (21h58m44.240627098s) [upload=21028]: Uploading segments... Jul 25 09:34:01.396495420 (21h58m44.241296955s) [upload=21028]: Got next segment. Inline: false Jul 25 09:34:01.396604095 (21h58m44.241405672s) [upload=21028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:34:01.396612168 (21h58m44.241413704s) [upload=21028]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:34:01.425826935 (21h58m44.270628471s) [seg_pos={0 0}, upload=21028]: Begin upload segment... Jul 25 09:34:01.426020430 (21h58m44.270821966s) [seg_pos={0 0}, upload=21028]: Done begin upload segment. Jul 25 09:34:01.426040728 (21h58m44.270842262s) [noise=true , address=104.36.82.4:4801 , piece_id=3NQQNM47LKMBO5GAQTMIWFTCGMD74JPMV4UPN7DTSVEKEZDZV5CA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.426061340 (21h58m44.2708629s) [noise=true , address=45.91.116.9:4263 , piece_id=VQVHJLCTXK5RMM2COSMGLP3UERPAFXL7CGCXROLSTIS4MIDTXGJA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.426601379 (21h58m44.271402918s) [noise=true , address=45.91.116.13:4155 , piece_id=V522D5F5OHME2REAEHIKBXKB4TA7FA24AOMZW5SMZV65UG4KW3XQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.426656014 (21h58m44.271457552s) [noise=true , address=77.90.31.13:4627 , piece_id=4AQ2PIDAK475LLLAG63ECCYHSZYWYVI52EJG7K5L7PA5BNBICA2A, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.427123147 (21h58m44.271924683s) [noise=true , address=45.91.116.7:4010 , piece_id=5OIKMU4MAE77EW635NMFJ7DOZNTCPC3QEYK52P5VITJJEZWFKEEA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.427565449 (21h58m44.272366988s) [noise=true , address=45.91.116.10:4083 , piece_id=D4PFM5EKLWNM5U3WVVHT5BVFXHPPNRFS64FEYLZORCR63FVQOEIQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.428074025 (21h58m44.272875567s) [noise=true , address=45.91.116.13:4154 , piece_id=OI6I6PDJKSYMDPFX4Q7AUWFRUXKMAZM7KI2X5QKTRHU3ZLYYCMVQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.428510546 (21h58m44.273312077s) [noise=true , address=77.90.31.16:4735 , piece_id=6ZY3FR55LO6O5E56LW6I5I5R7CHPC2MGS7NUJDD7UAXLOS3GXK4Q, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.428934503 (21h58m44.273736038s) [noise=true , address=45.91.116.6:4975 , piece_id=MI33ERQ7USAD2VE72XW5GV7G7HVUTDWTMUIXZQIBQQYOEHLR2DKQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.429488020 (21h58m44.274289548s) [noise=true , address=104.36.82.2:4730 , piece_id=VJNUI5MBLNWTMXZPXZNCFTSS4HBH7SBRXMQSMS36TIJOVLMA6X7Q, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.430016372 (21h58m44.274817913s) [noise=true , address=77.90.31.14:4661 , piece_id=AVBIPY5PRBUYUDVPMXCUFNUDQ6KPGDFMFW4HYZGUFLAKNC5ZLXYA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.430716875 (21h58m44.275518412s) [noise=true , address=104.36.82.12:4149 , piece_id=25LO6D4NKMKRLQQ4JNAXOLO5VAYANARND6RPNP5AW6J2WD5JCDXQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.431223998 (21h58m44.27602553s) [noise=true , address=77.90.31.19:4842 , piece_id=QWKQ455RQK2Q7IS3U5EXHAW3FX33XWJDTEV2GPDDAL5FJTEGTPGQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.431734863 (21h58m44.2765364s) [noise=true , address=104.36.82.14:4223 , piece_id=KG47KSNPFPH7PHYPLBC4GONV5XLYDG6M3HUMWZLK7G7PU6HPCXQQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.432299394 (21h58m44.277100922s) [noise=true , address=45.91.116.4:4937 , piece_id=GYYBC7JJCP2VOENJJAZRDAO6NMHVZXZEXA5M3APOATJCXDQ5XSNA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.432742241 (21h58m44.277543768s) [noise=true , address=77.90.31.15:4698 , piece_id=UKE2SEHPFX367ON5N2THRO5XN26KWOMQLP7RD36K2H5JTLXIRWOA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.541163047 (21h58m44.385964577s) [upload=21028]: Next returned nil segment Jul 25 09:34:01.541183693 (21h58m44.385985227s) [upload=21028]: Waiting for error group managing segments... Jul 25 09:34:01.555640486 (21h58m44.400441996s) [noise=true , address=45.91.116.6:4975 , piece_id=MI33ERQ7USAD2VE72XW5GV7G7HVUTDWTMUIXZQIBQQYOEHLR2DKQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.556377594 (21h58m44.401179119s) [noise=true , address=45.91.116.10:4083 , piece_id=D4PFM5EKLWNM5U3WVVHT5BVFXHPPNRFS64FEYLZORCR63FVQOEIQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.557556943 (21h58m44.402358454s) [noise=true , address=45.91.116.13:4155 , piece_id=V522D5F5OHME2REAEHIKBXKB4TA7FA24AOMZW5SMZV65UG4KW3XQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.558013379 (21h58m44.402814904s) [noise=true , address=77.90.31.13:4627 , piece_id=4AQ2PIDAK475LLLAG63ECCYHSZYWYVI52EJG7K5L7PA5BNBICA2A, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.558428945 (21h58m44.403230459s) [noise=true , address=45.91.116.9:4263 , piece_id=VQVHJLCTXK5RMM2COSMGLP3UERPAFXL7CGCXROLSTIS4MIDTXGJA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.558956623 (21h58m44.403758141s) [noise=true , address=104.36.82.2:4730 , piece_id=VJNUI5MBLNWTMXZPXZNCFTSS4HBH7SBRXMQSMS36TIJOVLMA6X7Q, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.559007198 (21h58m44.403808715s) [noise=true , address=45.91.116.13:4154 , piece_id=OI6I6PDJKSYMDPFX4Q7AUWFRUXKMAZM7KI2X5QKTRHU3ZLYYCMVQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.559590455 (21h58m44.404391965s) [noise=true , address=45.91.116.4:4937 , piece_id=GYYBC7JJCP2VOENJJAZRDAO6NMHVZXZEXA5M3APOATJCXDQ5XSNA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.559696005 (21h58m44.404497534s) [noise=true , address=77.90.31.14:4661 , piece_id=AVBIPY5PRBUYUDVPMXCUFNUDQ6KPGDFMFW4HYZGUFLAKNC5ZLXYA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.560471920 (21h58m44.405273433s) [noise=true , address=104.36.82.14:4223 , piece_id=KG47KSNPFPH7PHYPLBC4GONV5XLYDG6M3HUMWZLK7G7PU6HPCXQQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561137208 (21h58m44.405938736s) [noise=true , address=77.90.31.15:4698 , piece_id=UKE2SEHPFX367ON5N2THRO5XN26KWOMQLP7RD36K2H5JTLXIRWOA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561216161 (21h58m44.406017672s) [noise=true , address=77.90.31.19:4842 , piece_id=QWKQ455RQK2Q7IS3U5EXHAW3FX33XWJDTEV2GPDDAL5FJTEGTPGQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561662523 (21h58m44.40646404s) [noise=true , address=45.91.116.7:4010 , piece_id=5OIKMU4MAE77EW635NMFJ7DOZNTCPC3QEYK52P5VITJJEZWFKEEA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561675230 (21h58m44.406476756s) [seg_pos={0 0}, upload=21028]: Segment reached optimal threshold of 13 pieces. Jul 25 09:34:01.561800391 (21h58m44.406601905s) [noise=true , address=104.36.82.12:4149 , piece_id=25LO6D4NKMKRLQQ4JNAXOLO5VAYANARND6RPNP5AW6J2WD5JCDXQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:01.561994528 (21h58m44.406796058s) [noise=true , address=104.36.82.4:4801 , piece_id=3NQQNM47LKMBO5GAQTMIWFTCGMD74JPMV4UPN7DTSVEKEZDZV5CA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:01.562128894 (21h58m44.406930406s) [noise=true , address=77.90.31.16:4735 , piece_id=6ZY3FR55LO6O5E56LW6I5I5R7CHPC2MGS7NUJDD7UAXLOS3GXK4Q, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.562229276 (21h58m44.407030809s) [seg_pos={0 0}, upload=21028]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 25 09:34:01.562358118 (21h58m44.407159707s) [upload=21028]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:34:01.562365734 (21h58m44.407167258s) [upload=21028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:01.612443727 (21h58m44.457245259s) [upload=21028]: Done uploading segments... Jul 25 09:34:01.612489093 (21h58m44.457290622s) [upload=21028]: Upload finished. err: <nil> Duration: 461.887554ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 1.002393099s FAIL (2025-07-25 10:11:51 UTC) orbiter warning: ======ERROR====== uplink: permission denied (bucket: metaclient: Unauthorized API credentials) ======LOGS====== h item: pb.BatchRequestItem_SegmentBegin Jul 25 09:35:15.918314780 (21h59m58.763116293s) [seg_pos={0 0}, upload=21044]: Begin upload segment... Jul 25 09:35:15.918358070 (21h59m58.763159586s) [seg_pos={0 0}, upload=21044]: Done begin upload segment. Jul 25 09:35:15.918382251 (21h59m58.763183776s) [noise=true , address=77.90.31.14:4678 , piece_id=HIZS7J3DY6Q33PO2AI6UNXQ5S54L6DGG6ZAP3YKZZSEUINZCXEPQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.918407152 (21h59m58.763208685s) [noise=true , address=45.91.116.6:4994 , piece_id=JPWTQR5RKBZACY6XJDNQAEUCFDNYFLC7BYZCLHOZCCFJPXEKOSMA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.918842345 (21h59m58.763643878s) [noise=true , address=77.90.31.16:4751 , piece_id=DCODR5LGROHG7DVIUSRLFRRBKR672JGYQ3JOOTJSLL2NVLY6L62Q, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.918965591 (21h59m58.763767123s) [noise=true , address=77.90.31.15:4720 , piece_id=FL7PFV4B2TZ54WTT3YYU3NPGT7RBNGGHRUS4QTOARDZASB7PKAGQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.919180183 (21h59m58.763981708s) [noise=true , address=77.90.31.18:4830 , piece_id=2Z6ELONZADYWJJZPQUWKPTARWQI5D4CRE7XDIQBCUVVWAOUOYFGA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.919362724 (21h59m58.76416425s) [noise=true , address=79.127.205.235:4690 , piece_id=4MLADNII3WTUAMY5KKWUM2XSVFDR3NSLA32EMVJ3AIDOJENSF3SA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.919457169 (21h59m58.764258696s) [noise=true , address=79.127.205.246:4263 , piece_id=2RB4RONQXXPKZIETGCVEOM2JQUC5EIKETQA6VKCSSHF6ARLG2BRQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.919677786 (21h59m58.764479318s) [noise=true , address=104.36.82.15:4284 , piece_id=3OMO32MXTLOB6JUNQQVR27UNL5CHYN5XWWG3TO3DUOOIMEWYYY3A, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.919849288 (21h59m58.764650816s) [noise=true , address=45.91.116.10:4099 , piece_id=CWLXVFK56DYPPB3ED5INVEHKBUPW6MSKZLAA7SAFQU3YFT3UNHXQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.920171246 (21h59m58.764972769s) [noise=true , address=104.36.82.9:4021 , piece_id=T3KBEAMBTCD4I6UO52RTLVDFS47BRXJCZMGU44VEQITYQPM4K2YA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.920199227 (21h59m58.765000758s) [noise=true , address=104.36.82.50:4046 , piece_id=6UXAZY6MUTWLLVYZJBV4AYYIYH3KDR2IT22BWGGKBYJQGGMZGVIA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.920446438 (21h59m58.765247963s) [noise=true , address=77.90.31.19:4850 , piece_id=5FE4KIHO7B5CQ2ZHGB6CIZ4BA4ZYHMG666JB7S7YI3ZV25PM7DRQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.920500425 (21h59m58.765301948s) [noise=true , address=79.127.220.97:4367 , piece_id=466Y2CJB2PZ6EYNZLJAGIIXJHNAWS5P6L2GJXDWTYGA5CPVG2L3A, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.920773391 (21h59m58.765574929s) [noise=true , address=104.36.82.46:4509 , piece_id=LF6MNWECHJUQIIGTCYSQAMVWOUVYAJ7S7WPPANUYNDVVZPD26QCA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.920795516 (21h59m58.765597039s) [noise=true , address=79.127.219.47:4065 , piece_id=63NEETIH55O2QLL6OTFXXIGRE4MSCVSOSUKPNTCNHYENBJKPA5TQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.921069667 (21h59m58.76587119s) [noise=true , address=77.90.31.17:4790 , piece_id=PQGH3COTKNFS37FWGA6FLRVASZTELDOMRZVY5DGHTMVGZZIQMQWQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.921112088 (21h59m58.765913613s) [noise=true , address=79.127.201.215:4350 , piece_id=5V225ATIAYECLX2LXE2H5RZFJJCEAVTKIFCGKYWBOF532LOMGJJQ, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:15.921402936 (21h59m58.766204463s) [noise=true , address=45.91.116.7:4040 , piece_id=GFCCEQ5NJD4J7QZAXY7AEHZMCCRV3NHLNTK76YWW2TMOK46GKJQA, seg_pos={0 0}, upload=21044]: Uploading piece... Jul 25 09:35:16.026469477 (21h59m58.871270992s) [upload=21044]: Next returned nil segment Jul 25 09:35:16.026486428 (21h59m58.871287959s) [upload=21044]: Waiting for error group managing segments... Jul 25 09:35:16.032614751 (21h59m58.877416267s) [noise=true , address=77.90.31.19:4850 , piece_id=5FE4KIHO7B5CQ2ZHGB6CIZ4BA4ZYHMG666JB7S7YI3ZV25PM7DRQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.033106703 (21h59m58.877908219s) [noise=true , address=45.91.116.6:4994 , piece_id=JPWTQR5RKBZACY6XJDNQAEUCFDNYFLC7BYZCLHOZCCFJPXEKOSMA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.034162031 (21h59m58.87896354s) [noise=true , address=77.90.31.18:4830 , piece_id=2Z6ELONZADYWJJZPQUWKPTARWQI5D4CRE7XDIQBCUVVWAOUOYFGA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.036556969 (21h59m58.881358495s) [noise=true , address=104.36.82.46:4509 , piece_id=LF6MNWECHJUQIIGTCYSQAMVWOUVYAJ7S7WPPANUYNDVVZPD26QCA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.036598532 (21h59m58.881400048s) [noise=true , address=104.36.82.15:4284 , piece_id=3OMO32MXTLOB6JUNQQVR27UNL5CHYN5XWWG3TO3DUOOIMEWYYY3A, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.042836507 (21h59m58.887638032s) [noise=true , address=79.127.219.47:4065 , piece_id=63NEETIH55O2QLL6OTFXXIGRE4MSCVSOSUKPNTCNHYENBJKPA5TQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.046300094 (21h59m58.891101617s) [noise=true , address=77.90.31.14:4678 , piece_id=HIZS7J3DY6Q33PO2AI6UNXQ5S54L6DGG6ZAP3YKZZSEUINZCXEPQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.046770857 (21h59m58.891572381s) [noise=true , address=77.90.31.17:4790 , piece_id=PQGH3COTKNFS37FWGA6FLRVASZTELDOMRZVY5DGHTMVGZZIQMQWQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.047620050 (21h59m58.892421564s) [noise=true , address=104.36.82.9:4021 , piece_id=T3KBEAMBTCD4I6UO52RTLVDFS47BRXJCZMGU44VEQITYQPM4K2YA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.049041308 (21h59m58.893842842s) [noise=true , address=45.91.116.7:4040 , piece_id=GFCCEQ5NJD4J7QZAXY7AEHZMCCRV3NHLNTK76YWW2TMOK46GKJQA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.049699949 (21h59m58.894501458s) [noise=true , address=45.91.116.10:4099 , piece_id=CWLXVFK56DYPPB3ED5INVEHKBUPW6MSKZLAA7SAFQU3YFT3UNHXQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.058570793 (21h59m58.903372309s) [noise=true , address=104.36.82.50:4046 , piece_id=6UXAZY6MUTWLLVYZJBV4AYYIYH3KDR2IT22BWGGKBYJQGGMZGVIA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.063456832 (21h59m58.908258348s) [noise=true , address=77.90.31.16:4751 , piece_id=DCODR5LGROHG7DVIUSRLFRRBKR672JGYQ3JOOTJSLL2NVLY6L62Q, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.064332620 (21h59m58.909134133s) [noise=true , address=77.90.31.15:4720 , piece_id=FL7PFV4B2TZ54WTT3YYU3NPGT7RBNGGHRUS4QTOARDZASB7PKAGQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: <nil> Jul 25 09:35:16.064383494 (21h59m58.909185033s) [seg_pos={0 0}, upload=21044]: Segment reached optimal threshold of 14 pieces. Jul 25 09:35:16.064635168 (21h59m58.909436691s) [noise=true , address=79.127.220.97:4367 , piece_id=466Y2CJB2PZ6EYNZLJAGIIXJHNAWS5P6L2GJXDWTYGA5CPVG2L3A, seg_pos={0 0}, upload=21044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ukTBGQCf81aFiZhST9sqJEN28d1qg44gv27U2oPJKiGNirRPB): protocol: write tcp 159.223.143.39:49600->79.127.220.97:4367: use of closed network connection; write tcp 159.223.143.39:49600->79.127.220.97:4367: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:49600->79.127.220.97:4367: use of closed network connection Jul 25 09:35:16.064733953 (21h59m58.909535466s) [noise=true , address=79.127.205.235:4690 , piece_id=4MLADNII3WTUAMY5KKWUM2XSVFDR3NSLA32EMVJ3AIDOJENSF3SA, seg_pos={0 0}, upload=21044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121SG1z7mpp74ipZ9b7hVE9WrEicBpRk84uoc3JW7FKupxKLcdg): protocol: read tcp 159.223.143.39:49260->79.127.205.235:4690: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 25 09:35:16.064916028 (21h59m58.90971754s) [noise=true , address=79.127.201.215:4350 , piece_id=5V225ATIAYECLX2LXE2H5RZFJJCEAVTKIFCGKYWBOF532LOMGJJQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WhZAWvHCTj5xVkUuVZtgkFB1dhFKpBPq18YMKkepFx5qxkUAr): protocol: read tcp 159.223.143.39:39812->79.127.201.215:4350: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 25 09:35:16.064952221 (21h59m58.909753734s) [noise=true , address=79.127.205.246:4263 , piece_id=2RB4RONQXXPKZIETGCVEOM2JQUC5EIKETQA6VKCSSHF6ARLG2BRQ, seg_pos={0 0}, upload=21044]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZV35gnSB9uZZZQDr9v8esb7r4XH6TcxLdnC8LnESkoKxH2FZt): protocol: read tcp 159.223.143.39:54542->79.127.205.246:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 25 09:35:16.064963992 (21h59m58.909765516s) [seg_pos={0 0}, upload=21044]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 25 09:35:16.065052663 (21h59m58.909854191s) [upload=21044]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:35:16.065058796 (21h59m58.909860327s) [upload=21044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:35:16.170721369 (21h59m59.015522894s) [upload=21044]: Done uploading segments... Jul 25 09:35:16.170744580 (21h59m59.015546107s) [upload=21044]: Upload finished. err: <nil> Duration:* 1.002423119s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 717.240393ms FAIL (2025-07-25 10:11:49 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 717.273691ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................XX.........................................] duration: 590.813624ms FAIL (2025-07-25 10:11:40 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== oad=21019]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:40.996438590 (21h58m23.841240106s) [noise=true , address=77.90.31.13:4627 , piece_id=HBRLEALALTYTQHBRC3OM7VC2AN2D4GAXKF7M36TAF5HD7G6OTPBQ, seg_pos={0 0}, upload=21019]: Done uploading piece. err: <nil> Jul 25 09:33:40.996877017 (21h58m23.841678534s) [noise=true , address=77.90.31.17:4772 , piece_id=HZP632HLOH3CYRU3AZHCLZSOEX7BL4F22YGSO5NM6GY3YCIJ26PA, seg_pos={0 0}, upload=21019]: Done uploading piece. err: <nil> Jul 25 09:33:40.996900235 (21h58m23.841701752s) [seg_pos={0 0}, upload=21019]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil> Jul 25 09:33:40.996968719 (21h58m23.841770241s) [upload=21019]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:33:40.996975079 (21h58m23.841776601s) [upload=21019]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:41.050252368 (21h58m23.895054131s) [upload=21019]: Done uploading segments... Jul 25 09:33:41.050275330 (21h58m23.895076853s) [upload=21019]: Upload finished. err: <nil> Jul 25 09:34:01.395455482 (21h58m44.240257015s) [upload=21028]: Starting upload Jul 25 09:34:01.395607427 (21h58m44.240408952s) [upload=21028]: Done starting upload Jul 25 09:34:01.395825566 (21h58m44.240627098s) [upload=21028]: Uploading segments... Jul 25 09:34:01.396495420 (21h58m44.241296955s) [upload=21028]: Got next segment. Inline: false Jul 25 09:34:01.396604095 (21h58m44.241405672s) [upload=21028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:34:01.396612168 (21h58m44.241413704s) [upload=21028]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:34:01.425826935 (21h58m44.270628471s) [seg_pos={0 0}, upload=21028]: Begin upload segment... Jul 25 09:34:01.426020430 (21h58m44.270821966s) [seg_pos={0 0}, upload=21028]: Done begin upload segment. Jul 25 09:34:01.426040728 (21h58m44.270842262s) [noise=true , address=104.36.82.4:4801 , piece_id=3NQQNM47LKMBO5GAQTMIWFTCGMD74JPMV4UPN7DTSVEKEZDZV5CA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.426061340 (21h58m44.2708629s) [noise=true , address=45.91.116.9:4263 , piece_id=VQVHJLCTXK5RMM2COSMGLP3UERPAFXL7CGCXROLSTIS4MIDTXGJA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.426601379 (21h58m44.271402918s) [noise=true , address=45.91.116.13:4155 , piece_id=V522D5F5OHME2REAEHIKBXKB4TA7FA24AOMZW5SMZV65UG4KW3XQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.426656014 (21h58m44.271457552s) [noise=true , address=77.90.31.13:4627 , piece_id=4AQ2PIDAK475LLLAG63ECCYHSZYWYVI52EJG7K5L7PA5BNBICA2A, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.427123147 (21h58m44.271924683s) [noise=true , address=45.91.116.7:4010 , piece_id=5OIKMU4MAE77EW635NMFJ7DOZNTCPC3QEYK52P5VITJJEZWFKEEA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.427565449 (21h58m44.272366988s) [noise=true , address=45.91.116.10:4083 , piece_id=D4PFM5EKLWNM5U3WVVHT5BVFXHPPNRFS64FEYLZORCR63FVQOEIQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.428074025 (21h58m44.272875567s) [noise=true , address=45.91.116.13:4154 , piece_id=OI6I6PDJKSYMDPFX4Q7AUWFRUXKMAZM7KI2X5QKTRHU3ZLYYCMVQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.428510546 (21h58m44.273312077s) [noise=true , address=77.90.31.16:4735 , piece_id=6ZY3FR55LO6O5E56LW6I5I5R7CHPC2MGS7NUJDD7UAXLOS3GXK4Q, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.428934503 (21h58m44.273736038s) [noise=true , address=45.91.116.6:4975 , piece_id=MI33ERQ7USAD2VE72XW5GV7G7HVUTDWTMUIXZQIBQQYOEHLR2DKQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.429488020 (21h58m44.274289548s) [noise=true , address=104.36.82.2:4730 , piece_id=VJNUI5MBLNWTMXZPXZNCFTSS4HBH7SBRXMQSMS36TIJOVLMA6X7Q, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.430016372 (21h58m44.274817913s) [noise=true , address=77.90.31.14:4661 , piece_id=AVBIPY5PRBUYUDVPMXCUFNUDQ6KPGDFMFW4HYZGUFLAKNC5ZLXYA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.430716875 (21h58m44.275518412s) [noise=true , address=104.36.82.12:4149 , piece_id=25LO6D4NKMKRLQQ4JNAXOLO5VAYANARND6RPNP5AW6J2WD5JCDXQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.431223998 (21h58m44.27602553s) [noise=true , address=77.90.31.19:4842 , piece_id=QWKQ455RQK2Q7IS3U5EXHAW3FX33XWJDTEV2GPDDAL5FJTEGTPGQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.431734863 (21h58m44.2765364s) [noise=true , address=104.36.82.14:4223 , piece_id=KG47KSNPFPH7PHYPLBC4GONV5XLYDG6M3HUMWZLK7G7PU6HPCXQQ, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.432299394 (21h58m44.277100922s) [noise=true , address=45.91.116.4:4937 , piece_id=GYYBC7JJCP2VOENJJAZRDAO6NMHVZXZEXA5M3APOATJCXDQ5XSNA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.432742241 (21h58m44.277543768s) [noise=true , address=77.90.31.15:4698 , piece_id=UKE2SEHPFX367ON5N2THRO5XN26KWOMQLP7RD36K2H5JTLXIRWOA, seg_pos={0 0}, upload=21028]: Uploading piece... Jul 25 09:34:01.541163047 (21h58m44.385964577s) [upload=21028]: Next returned nil segment Jul 25 09:34:01.541183693 (21h58m44.385985227s) [upload=21028]: Waiting for error group managing segments... Jul 25 09:34:01.555640486 (21h58m44.400441996s) [noise=true , address=45.91.116.6:4975 , piece_id=MI33ERQ7USAD2VE72XW5GV7G7HVUTDWTMUIXZQIBQQYOEHLR2DKQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.556377594 (21h58m44.401179119s) [noise=true , address=45.91.116.10:4083 , piece_id=D4PFM5EKLWNM5U3WVVHT5BVFXHPPNRFS64FEYLZORCR63FVQOEIQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.557556943 (21h58m44.402358454s) [noise=true , address=45.91.116.13:4155 , piece_id=V522D5F5OHME2REAEHIKBXKB4TA7FA24AOMZW5SMZV65UG4KW3XQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.558013379 (21h58m44.402814904s) [noise=true , address=77.90.31.13:4627 , piece_id=4AQ2PIDAK475LLLAG63ECCYHSZYWYVI52EJG7K5L7PA5BNBICA2A, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.558428945 (21h58m44.403230459s) [noise=true , address=45.91.116.9:4263 , piece_id=VQVHJLCTXK5RMM2COSMGLP3UERPAFXL7CGCXROLSTIS4MIDTXGJA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.558956623 (21h58m44.403758141s) [noise=true , address=104.36.82.2:4730 , piece_id=VJNUI5MBLNWTMXZPXZNCFTSS4HBH7SBRXMQSMS36TIJOVLMA6X7Q, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.559007198 (21h58m44.403808715s) [noise=true , address=45.91.116.13:4154 , piece_id=OI6I6PDJKSYMDPFX4Q7AUWFRUXKMAZM7KI2X5QKTRHU3ZLYYCMVQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.559590455 (21h58m44.404391965s) [noise=true , address=45.91.116.4:4937 , piece_id=GYYBC7JJCP2VOENJJAZRDAO6NMHVZXZEXA5M3APOATJCXDQ5XSNA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.559696005 (21h58m44.404497534s) [noise=true , address=77.90.31.14:4661 , piece_id=AVBIPY5PRBUYUDVPMXCUFNUDQ6KPGDFMFW4HYZGUFLAKNC5ZLXYA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.560471920 (21h58m44.405273433s) [noise=true , address=104.36.82.14:4223 , piece_id=KG47KSNPFPH7PHYPLBC4GONV5XLYDG6M3HUMWZLK7G7PU6HPCXQQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561137208 (21h58m44.405938736s) [noise=true , address=77.90.31.15:4698 , piece_id=UKE2SEHPFX367ON5N2THRO5XN26KWOMQLP7RD36K2H5JTLXIRWOA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561216161 (21h58m44.406017672s) [noise=true , address=77.90.31.19:4842 , piece_id=QWKQ455RQK2Q7IS3U5EXHAW3FX33XWJDTEV2GPDDAL5FJTEGTPGQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561662523 (21h58m44.40646404s) [noise=true , address=45.91.116.7:4010 , piece_id=5OIKMU4MAE77EW635NMFJ7DOZNTCPC3QEYK52P5VITJJEZWFKEEA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.561675230 (21h58m44.406476756s) [seg_pos={0 0}, upload=21028]: Segment reached optimal threshold of 13 pieces. Jul 25 09:34:01.561800391 (21h58m44.406601905s) [noise=true , address=104.36.82.12:4149 , piece_id=25LO6D4NKMKRLQQ4JNAXOLO5VAYANARND6RPNP5AW6J2WD5JCDXQ, seg_pos={0 0}, upload=21028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:01.561994528 (21h58m44.406796058s) [noise=true , address=104.36.82.4:4801 , piece_id=3NQQNM47LKMBO5GAQTMIWFTCGMD74JPMV4UPN7DTSVEKEZDZV5CA, seg_pos={0 0}, upload=21028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:01.562128894 (21h58m44.406930406s) [noise=true , address=77.90.31.16:4735 , piece_id=6ZY3FR55LO6O5E56LW6I5I5R7CHPC2MGS7NUJDD7UAXLOS3GXK4Q, seg_pos={0 0}, upload=21028]: Done uploading piece. err: <nil> Jul 25 09:34:01.562229276 (21h58m44.407030809s) [seg_pos={0 0}, upload=21028]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 25 09:34:01.562358118 (21h58m44.407159707s) [upload=21028]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:34:01.562365734 (21h58m44.407167258s) [upload=21028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:01.612443727 (21h58m44.457245259s) [upload=21028]: Done uploading segments... Jul 25 09:34:01.612489093 (21h58m44.457290622s) [upload=21028]: Upload finished. err: <nil> Duration: 590.844729ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 534.42164ms FAIL (2025-07-25 10:11:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 403, RequestID: 185574E861CBB3B7, HostID: , api error AccessDenied: Access Denied. Duration: 534.448355ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [............X............X...X..............................] duration: 269.453685ms FAIL (2025-07-25 10:11:31 UTC) orbiter warning: uplink: invalid range 0:1024 (size:1024) Duration: 269.501348ms core uplink us1 vivint

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)