Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [X......X....................................................] duration: 809.142402ms FAIL (2025-08-07 17:59:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 809.182897ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 277.953054ms FAIL (2025-08-07 17:59:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 278.012865ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 235.996641ms FAIL (2025-08-07 17:59:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 236.026962ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X......X...................................................] duration: 450.254924ms FAIL (2025-08-07 17:58:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 450.36062ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 274.512533ms FAIL (2025-08-07 17:57:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 274.543787ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X......X..................................................] duration: 463.649049ms FAIL (2025-08-07 17:57:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 463.758423ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 332.710906ms FAIL (2025-08-07 17:57:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 332.738117ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X......X.................................................] duration: 533.29118ms FAIL (2025-08-07 17:57:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 533.313791ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 190.199002ms FAIL (2025-08-07 17:56:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 190.226877ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 1.714934179s FAIL (2025-08-07 17:56:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 1.714957905s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [....X......X................................................] duration: 748.206944ms FAIL (2025-08-07 17:55:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 748.239655ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 670.46606ms FAIL (2025-08-07 17:55:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 670.492096ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....X......X...............................................] duration: 699.140843ms FAIL (2025-08-07 17:55:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 699.164285ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [......X......X..............................................] duration: 561.320585ms FAIL (2025-08-07 17:54:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 561.351884ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 479.024248ms FAIL (2025-08-07 17:54:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 479.075642ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......X......X.............................................] duration: 1.179042341s FAIL (2025-08-07 17:53:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 1.179069707s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [........X......X............................................] duration: 622.27537ms FAIL (2025-08-07 17:53:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 622.29962ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 228.105707ms FAIL (2025-08-07 17:53:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 228.135635ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 231.082586ms FAIL (2025-08-07 17:52:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 231.108599ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........X......X...........................................] duration: 479.361986ms FAIL (2025-08-07 17:52:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 479.392799ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........X......X..........................................] duration: 712.734241ms FAIL (2025-08-07 17:52:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 712.757178ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 226.792556ms FAIL (2025-08-07 17:51:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 226.82329ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........X......X.........................................] duration: 521.692258ms FAIL (2025-08-07 17:51:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 521.735185ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 260.336566ms FAIL (2025-08-07 17:51:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 260.426998ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [............X......X........................................] duration: 659.012166ms FAIL (2025-08-07 17:50:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 659.037552ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 189.427276ms FAIL (2025-08-07 17:50:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 189.45587ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 201.151769ms FAIL (2025-08-07 17:49:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 201.198856ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............X......X.......................................] duration: 429.807942ms FAIL (2025-08-07 17:49:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 429.837179ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 527.259911ms FAIL (2025-08-07 17:48:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 527.325908ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............X......X......................................] duration: 1.185531446s FAIL (2025-08-07 17:48:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 1.185565393s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............X......X.....................................] duration: 671.694615ms FAIL (2025-08-07 17:48:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 671.728615ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 509.651344ms FAIL (2025-08-07 17:47:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 509.67833ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [................X......X....................................] duration: 858.631367ms FAIL (2025-08-07 17:47:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 858.662748ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 390.313493ms FAIL (2025-08-07 17:47:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 390.348807ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 295.591202ms FAIL (2025-08-07 17:46:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 295.616522ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................X......X...................................] duration: 519.766984ms FAIL (2025-08-07 17:45:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 519.791459ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 308.750014ms FAIL (2025-08-07 17:45:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 308.773582ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................X......X..................................] duration: 529.367161ms FAIL (2025-08-07 17:45:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 529.395761ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 390.148717ms FAIL (2025-08-07 17:44:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 390.175774ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 368.348157ms FAIL (2025-08-07 17:44:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 368.371879ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................X......X.................................] duration: 837.478066ms FAIL (2025-08-07 17:44:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 837.508955ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [X...................X......X................................] duration: 937.988316ms FAIL (2025-08-07 17:43:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 938.044607ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 419.533885ms FAIL (2025-08-07 17:43:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 419.566407ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 208.226753ms FAIL (2025-08-07 17:43:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 208.252798ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.X...................X......X...............................] duration: 682.231337ms FAIL (2025-08-07 17:42:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 682.254773ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 221.688283ms FAIL (2025-08-07 17:42:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 221.715211ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..X...................X......X..............................] duration: 688.386069ms FAIL (2025-08-07 17:42:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 688.411249ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...X...................X......X.............................] duration: 565.620363ms FAIL (2025-08-07 17:41:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 565.644541ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 325.567564ms FAIL (2025-08-07 17:41:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== uestItem_SegmentMakeInline Aug 7 17:18:14.373232227 (7h38m48.888334814s) [upload=7942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:18:14.633255770 (7h38m49.148358343s) [upload=7942]: Done uploading segments... Aug 7 17:18:14.633279311 (7h38m49.148381894s) [upload=7942]: Upload finished. err: <nil> Aug 7 17:19:18.125473157 (7h39m52.640575747s) [upload=7958]: Starting upload Aug 7 17:19:18.125533674 (7h39m52.640636432s) [upload=7958]: Done starting upload Aug 7 17:19:18.125842443 (7h39m52.640945012s) [upload=7958]: Uploading segments... Aug 7 17:19:18.125899636 (7h39m52.64100222s) [upload=7958]: Got next segment. Inline: true Aug 7 17:19:18.125905979 (7h39m52.641008556s) [upload=7958]: Waiting for error group managing segments... Aug 7 17:19:18.125950645 (7h39m52.641053221s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:18.125955794 (7h39m52.641058428s) [upload=7958]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:19:18.125959555 (7h39m52.64106212s) [upload=7958]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:18.242447746 (7h39m52.757550324s) [upload=7958]: Done uploading segments... Aug 7 17:19:18.242466363 (7h39m52.757568938s) [upload=7958]: Upload finished. err: <nil> Aug 7 17:20:19.551585421 (7h40m54.066688004s) [upload=7976]: Starting upload Aug 7 17:20:19.551681243 (7h40m54.066783825s) [upload=7976]: Done starting upload Aug 7 17:20:19.551822845 (7h40m54.066925413s) [upload=7976]: Uploading segments... Aug 7 17:20:19.551888030 (7h40m54.066990613s) [upload=7976]: Got next segment. Inline: true Aug 7 17:20:19.551894389 (7h40m54.066996956s) [upload=7976]: Waiting for error group managing segments... Aug 7 17:20:19.551961190 (7h40m54.067063768s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:20:19.552136618 (7h40m54.067239202s) [upload=7976]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:20:19.552142476 (7h40m54.067245054s) [upload=7976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:20:19.835607561 (7h40m54.350710137s) [upload=7976]: Done uploading segments... Aug 7 17:20:19.835627405 (7h40m54.350729982s) [upload=7976]: Upload finished. err: <nil> Aug 7 17:21:00.679453492 (7h41m35.194556058s) [upload=7989]: Starting upload Aug 7 17:21:00.679507397 (7h41m35.194609973s) [upload=7989]: Done starting upload Aug 7 17:21:00.679600428 (7h41m35.194703005s) [upload=7989]: Uploading segments... Aug 7 17:21:00.679669966 (7h41m35.194772551s) [upload=7989]: Got next segment. Inline: true Aug 7 17:21:00.679676001 (7h41m35.194778578s) [upload=7989]: Waiting for error group managing segments... Aug 7 17:21:00.679736947 (7h41m35.194839523s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:00.679742209 (7h41m35.194844785s) [upload=7989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:00.679745465 (7h41m35.194848041s) [upload=7989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:21:00.776905685 (7h41m35.292008261s) [upload=7989]: Done uploading segments... Aug 7 17:21:00.776926714 (7h41m35.292029279s) [upload=7989]: Upload finished. err: <nil> Aug 7 17:21:59.906590795 (7h42m34.421693357s) [upload=8004]: Starting upload Aug 7 17:21:59.906636626 (7h42m34.421739202s) [upload=8004]: Done starting upload Aug 7 17:21:59.906691540 (7h42m34.421794117s) [upload=8004]: Uploading segments... Aug 7 17:21:59.906725786 (7h42m34.421828372s) [upload=8004]: Got next segment. Inline: true Aug 7 17:21:59.906729628 (7h42m34.421832204s) [upload=8004]: Waiting for error group managing segments... Aug 7 17:21:59.906765277 (7h42m34.421867854s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:21:59.906769243 (7h42m34.42187182s) [upload=8004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:21:59.906771557 (7h42m34.421874134s) [upload=8004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:00.028262172 (7h42m34.54336475s) [upload=8004]: Done uploading segments... Aug 7 17:22:00.028282801 (7h42m34.543385379s) [upload=8004]: Upload finished. err: <nil> Aug 7 17:22:56.860357310 (7h43m31.375459889s) [upload=8018]: Starting upload Aug 7 17:22:56.860405149 (7h43m31.375507728s) [upload=8018]: Done starting upload Aug 7 17:22:56.860472593 (7h43m31.375575161s) [upload=8018]: Uploading segments... Aug 7 17:22:56.860523453 (7h43m31.375626038s) [upload=8018]: Got next segment. Inline: true Aug 7 17:22:56.860529001 (7h43m31.375631577s) [upload=8018]: Waiting for error group managing segments... Aug 7 17:22:56.860569340 (7h43m31.375671908s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:22:56.860584432 (7h43m31.375687011s) [upload=8018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:22:56.860587673 (7h43m31.37569025s) [upload=8018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:22:56.954592787 (7h43m31.469695364s) [upload=8018]: Done uploading segments... Aug 7 17:22:56.954613376 (7h43m31.469715956s) [upload=8018]: Upload finished. err: <nil> Aug 7 17:23:29.078197372 (7h44m3.593299953s) [upload=8025]: Starting upload Aug 7 17:23:29.078295675 (7h44m3.593398261s) [upload=8025]: Done starting upload Aug 7 17:23:29.078425054 (7h44m3.593527632s) [upload=8025]: Uploading segments... Aug 7 17:23:29.078517549 (7h44m3.593620133s) [upload=8025]: Got next segment. Inline: true Aug 7 17:23:29.078527467 (7h44m3.593630053s) [upload=8025]: Waiting for error group managing segments... Aug 7 17:23:29.078595582 (7h44m3.59369817s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:23:29.078600856 (7h44m3.593703444s) [upload=8025]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:23:29.078604890 (7h44m3.593707468s) [upload=8025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:23:29.258549120 (7h44m3.773651701s) [upload=8025]: Done uploading segments... Aug 7 17:23:29.258569906 (7h44m3.773672471s) [upload=8025]: Upload finished. err: <nil> Aug 7 17:24:20.600890722 (7h44m55.115993308s) [upload=8044]: Starting upload Aug 7 17:24:20.600967079 (7h44m55.116069657s) [upload=8044]: Done starting upload Aug 7 17:24:20.601832161 (7h44m55.11693473s) [upload=8044]: Uploading segments... Aug 7 17:24:20.601998051 (7h44m55.117100642s) [upload=8044]: Got next segment. Inline: true Aug 7 17:24:20.602009046 (7h44m55.117111625s) [upload=8044]: Waiting for error group managing segments... Aug 7 17:24:20.602173006 (7h44m55.117275588s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:24:20.602267729 (7h44m55.117370305s) [upload=8044]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:24:20.602271926 (7h44m55.117374502s) [upload=8044]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:24:20.698684640 (7h44m55.21378724s) [upload=8044]: Done uploading segments... Aug 7 17:24:20.698733374 (7h44m55.213835961s) [upload=8044]: Upload finished. err: <nil> Aug 7 17:25:18.277692036 (7h45m52.792794612s) [upload=8061]: Starting upload Aug 7 17:25:18.277780789 (7h45m52.792883374s) [upload=8061]: Done starting upload Aug 7 17:25:18.277878751 (7h45m52.79298133s) [upload=8061]: Uploading segments... Aug 7 17:25:18.277935850 (7h45m52.793038436s) [upload=8061]: Got next segment. Inline: true Aug 7 17:25:18.277941085 (7h45m52.793043662s) [upload=8061]: Waiting for error group managing segments... Aug 7 17:25:18.277993969 (7h45m52.793096545s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:18.277998975 (7h45m52.793101552s) [upload=8061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:18.278002684 (7h45m52.793105261s) [upload=8061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:18.464114177 (7h45m52.97921677s) [upload=8061]: Done uploading segments... Aug 7 17:25:18.464144330 (7h45m52.979246913s) [upload=8061]: Upload finished. err: <nil> Aug 7 17:25:35.823492457 (7h46m10.338595038s) [upload=8066]: Starting upload Aug 7 17:25:35.823582908 (7h46m10.338685495s) [upload=8066]: Done starting upload Aug 7 17:25:35.823728442 (7h46m10.338831011s) [upload=8066]: Uploading segments... Aug 7 17:25:35.823882629 (7h46m10.338985213s) [upload=8066]: Got next segment. Inline: true Aug 7 17:25:35.823890460 (7h46m10.338993042s) [upload=8066]: Waiting for error group managing segments... Aug 7 17:25:35.824093967 (7h46m10.339196766s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:25:35.824101730 (7h46m10.339204313s) [upload=8066]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:25:35.824105926 (7h46m10.339208507s) [upload=8066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:25:35.967881517 (7h46m10.482984095s) [upload=8066]: Done uploading segments... Aug 7 17:25:35.967896043 (7h46m10.482998612s) [upload=8066]: Upload finished. err: <nil> Aug 7 17:26:37.546225040 (7h47m12.061327617s) [upload=8082]: Starting upload Aug 7 17:26:37.546264653 (7h47m12.061367238s) [upload=8082]: Done starting upload Aug 7 17:26:37.546331215 (7h47m12.061433784s) [upload=8082]: Uploading segments... Aug 7 17:26:37.546378757 (7h47m12.061481341s) [upload=8082]: Got next segment. Inline: true Aug 7 17:26:37.546383524 (7h47m12.061486101s) [upload=8082]: Waiting for error group managing segments... Aug 7 17:26:37.546423285 (7h47m12.061525862s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:26:37.546438329 (7h47m12.061540906s) [upload=8082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:26:37.546444671 (7h47m12.061547254s) [upload=8082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:26:37.690955638 (7h47m12.206058203s) [upload=8082]: Done uploading segments... Aug 7 17:26:37.690974188 (7h47m12.206076755s) [upload=8082]: Upload finished. err: <nil> Duration:* 325.594902ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [....X...................X......X............................] duration: 469.37805ms FAIL (2025-08-07 17:41:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== ading piece... Aug 7 17:19:12.833748995 (7h39m47.348851583s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834126787 (7h39m47.34922937s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834383760 (7h39m47.349486345s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834485062 (7h39m47.349587639s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834766235 (7h39m47.349868814s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.834793894 (7h39m47.349896473s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835125272 (7h39m47.350227861s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835142582 (7h39m47.350245147s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835530984 (7h39m47.350633566s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835565280 (7h39m47.350667866s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835878675 (7h39m47.350981255s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.835928932 (7h39m47.351031511s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836349386 (7h39m47.351451973s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836684887 (7h39m47.351787466s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.836995987 (7h39m47.352098573s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837335261 (7h39m47.35243784s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.837695820 (7h39m47.352798404s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Uploading piece... Aug 7 17:19:12.953105173 (7h39m47.468207749s) [upload=7955]: Next returned nil segment Aug 7 17:19:12.953117571 (7h39m47.468220148s) [upload=7955]: Waiting for error group managing segments... Aug 7 17:19:12.960698357 (7h39m47.475800935s) [noise=true , address=104.36.82.9:4020 , piece_id=B7EQX7MFZXBOX7CTJBEN2IOGUSF7QZ7UKWHACFKHHUT53XNEQSWA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.964637666 (7h39m47.479740234s) [noise=true , address=104.36.82.55:4224 , piece_id=MSM7U4JDYUD5JFQK4IVILQX5YF3FW6KZD6XQOXIB25FP3GUJ7C3Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965128570 (7h39m47.480231147s) [noise=true , address=104.36.82.47:4528 , piece_id=OST7DAPQRWH3MZI5YFKUXZAT3FJ7T5CNTFVEQGJCZ635R4G2D4VQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.965234427 (7h39m47.480336996s) [noise=true , address=104.36.82.61:4395 , piece_id=RPK7XZ5WFAE2H2BNCSFQOZHDZVUIZTA473ICBQBTCIAFNHEJWVLA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.968627617 (7h39m47.483730187s) [noise=true , address=104.36.82.37:4683 , piece_id=3IGSEELEFUFMDUJBIMMTDRMXBOYRE4YO56WIVFHHPYYHGWGLLRIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.973343324 (7h39m47.488445893s) [noise=true , address=104.36.82.62:4426 , piece_id=PV66CW62YGXPOUHSF3OJOHDQ4LGTUCPO3RW72PLJZYEPWLPLBUCQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.979560404 (7h39m47.494662981s) [noise=true , address=104.36.82.6:4907 , piece_id=XYKTV5OYQKRHPGRHQBWIYIHHX3RXP6VYSWPPDGSN4V6HMDI46DIQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.981142260 (7h39m47.496244856s) [noise=true , address=104.36.82.44:4894 , piece_id=IEGSN7POR3UPEINWSSNRKKNKW4TLIOYP4LUDVZAVH3IQZZZIO27A, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985148313 (7h39m47.500250882s) [noise=true , address=104.36.82.42:4797 , piece_id=5L7URCLMLZHHIR62QCDTINGM4YCGHIY3323ZQDNKOQL22RUVBV6Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985431306 (7h39m47.500533883s) [noise=true , address=104.36.82.16:4304 , piece_id=ULQJV4VFFJJGYMORJ4XUH5LMIOYI64EQJGUCM7ZYD3AZQVGQIMNA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.985809127 (7h39m47.500911704s) [noise=true , address=104.36.82.58:4289 , piece_id=ZEOKCI57TSGA2INEQQEYTTEPBMWDEWG4ANYFDRHXSEANG65IIQXQ, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:12.987931453 (7h39m47.50303402s) [noise=true , address=77.90.31.18:4827 , piece_id=D3NTXRT7USHG3BZHCRQ66DCHG5B4QTBFHVIVBDU5JTAPNRD2UMDA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.011562045 (7h39m47.526664611s) [noise=true , address=104.36.82.38:4717 , piece_id=QY3R3YC6WHIJD4MHYYMDBP4IUNNM5VUIAFSOQTX273D3BKBLCT7Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015177134 (7h39m47.530279713s) [noise=true , address=104.36.82.46:4497 , piece_id=XV7EHJ4VPODUDE2XZJYQPVDIZZJVJ4FQLW2R363U7BYG2EOYSZCA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: <nil> Aug 7 17:19:13.015226585 (7h39m47.530329162s) [seg_pos={0 0}, upload=7955]: Segment reached optimal threshold of 14 pieces. Aug 7 17:19:13.015459637 (7h39m47.530562214s) [noise=true , address=104.36.82.52:4116 , piece_id=XRSAROW5WPKGHX6IDJ6D4FQEOILP5DP7S5SDXQE3L6BBHHYLUKJA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vKCXgWTAsipTvUK9XA3sL6DznFA8BqZc6WCN84PBuLhKCTT9m): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015515480 (7h39m47.530618043s) [noise=true , address=104.36.82.10:4097 , piece_id=5O3TN5RLRKSYBBYG6J5ER4WWSIZJWGCCRF2VY2U6FCKX2H5Q2MQA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QP2aZ9BC22cD1Nzz2E12DiA1XoNu3GGTgHC8Cc3TvpQM3Ubn3): protocol: expected piece hash; context canceled; EOF Aug 7 17:19:13.015721682 (7h39m47.530824259s) [noise=true , address=79.127.201.215:4357 , piece_id=VHLOHJ47THJMQNJPODSWE7PY2TF5ELAY4X5KFUUY44BLGBBPPLUA, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yXKsyPEVG4W52L7SUmsDGt4oD2D9YyqsfwDVWpjMnMnyVo2Da): protocol: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:42430->79.127.201.215:4357: use of closed network connection Aug 7 17:19:13.015861474 (7h39m47.530964051s) [noise=true , address=79.127.205.233:4631 , piece_id=4LSZXQ7JFZLEVGI2RA6KWLRLEA6ZEZW6AVZADBTI7M2YYMEK7R2Q, seg_pos={0 0}, upload=7955]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tNcNixo72dz84o6bJjxoAnDByxLeHwEDBBwNrE3R5MvTCRr4v): protocol: read tcp 159.223.143.39:53758->79.127.205.233:4631: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 7 17:19:13.015889836 (7h39m47.530992401s) [seg_pos={0 0}, upload=7955]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:19:13.016110931 (7h39m47.53121351s) [upload=7955]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:19:13.016116825 (7h39m47.531219402s) [upload=7955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:19:13.120652413 (7h39m47.635754992s) [upload=7955]: Done uploading segments... Aug 7 17:19:13.120669774 (7h39m47.63577235s) [upload=7955]: Upload finished. err: <nil> Aug 7 17:19:42.864259295 (7h40m17.37936188s) [upload=7965]: Starting upload Aug 7 17:19:42.864327115 (7h40m17.379429698s) [upload=7965]: Done starting upload Aug 7 17:19:42.864430437 (7h40m17.379533013s) [upload=7965]: Uploading segments... Aug 7 17:19:42.864580867 (7h40m17.379683436s) [upload=7965]: Got next segment. Inline: false Aug 7 17:19:42.864646734 (7h40m17.37974931s) [upload=7965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:19:42.864651623 (7h40m17.3797542s) [upload=7965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:19:42.872995214 (7h40m17.388097804s) [upload=7965]: Next segment err: metaclient: Too Many Requests Aug 7 17:19:42.873032700 (7h40m17.388135278s) [upload=7965]: Duration: 469.418273ms core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)