Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
208.866921ms
FAIL (2025-08-08 11:02:21 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 208.914877ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
209.379409ms
FAIL (2025-08-08 11:01:11 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 209.408826ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
235.151364ms
FAIL (2025-08-08 11:00:12 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 235.188409ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
187.639649ms
FAIL (2025-08-08 10:59:57 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 187.668164ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
195.245232ms
FAIL (2025-08-08 10:58:56 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 195.270105ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
186.599685ms
FAIL (2025-08-08 10:58:36 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 186.628855ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
196.265908ms
FAIL (2025-08-08 10:57:46 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 196.290429ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
195.187053ms
FAIL (2025-08-08 10:57:13 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 195.211379ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
193.351394ms
FAIL (2025-08-08 10:56:00 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 193.384973ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
200.268092ms
FAIL (2025-08-08 10:54:57 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 200.293997ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
208.707164ms
FAIL (2025-08-08 10:53:44 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 208.736416ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
187.158443ms
FAIL (2025-08-08 10:53:17 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 187.185423ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
202.646036ms
FAIL (2025-08-08 10:52:45 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 202.671634ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
187.233069ms
FAIL (2025-08-08 10:51:50 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 187.263491ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
206.8792ms
FAIL (2025-08-08 10:51:25 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 206.904142ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
220.77582ms
FAIL (2025-08-08 10:50:38 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 220.82002ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
228.142496ms
FAIL (2025-08-08 10:50:21 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 228.171925ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
210.599509ms
FAIL (2025-08-08 10:50:00 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 210.6247ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
183.707439ms
FAIL (2025-08-08 10:49:10 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 183.738115ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
180.881286ms
FAIL (2025-08-08 10:48:26 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 180.911015ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
214.285276ms
FAIL (2025-08-08 10:47:18 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 214.309758ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
212.189931ms
FAIL (2025-08-08 10:46:09 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 212.212937ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
174.63505ms
FAIL (2025-08-08 10:45:33 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 174.791768ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
184.283236ms
FAIL (2025-08-08 10:44:48 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 184.310218ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
179.745225ms
FAIL (2025-08-08 10:44:18 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 179.765061ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
205.550382ms
FAIL (2025-08-08 10:43:27 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 205.583672ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
236.058353ms
FAIL (2025-08-08 10:42:56 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 236.0821ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
180.962214ms
FAIL (2025-08-08 10:41:58 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 180.991675ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
454.039265ms
FAIL (2025-08-08 10:40:59 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 454.068592ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
290.980187ms
FAIL (2025-08-08 10:40:30 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 291.003014ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
364.949207ms
FAIL (2025-08-08 10:39:59 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 364.975153ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
216.141779ms
FAIL (2025-08-08 10:39:25 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 216.171015ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
179.348112ms
FAIL (2025-08-08 10:38:51 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 179.380485ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
197.799163ms
FAIL (2025-08-08 10:38:10 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 197.826715ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
418.722458ms
FAIL (2025-08-08 10:37:24 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 418.748368ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
244.234342ms
FAIL (2025-08-08 10:36:41 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 244.262011ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
418.541403ms
FAIL (2025-08-08 10:36:19 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 418.566363ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
188.801965ms
FAIL (2025-08-08 10:35:27 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 188.835549ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
198.645118ms
FAIL (2025-08-08 10:34:48 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 198.690466ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
204.326296ms
FAIL (2025-08-08 10:34:15 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 204.353515ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
188.175703ms
FAIL (2025-08-08 10:33:07 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 188.203299ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
356.061982ms
FAIL (2025-08-08 10:31:59 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 356.108914ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
186.067081ms
FAIL (2025-08-08 10:31:21 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 186.095896ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
206.983831ms
FAIL (2025-08-08 10:31:00 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 207.009422ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
205.501265ms
FAIL (2025-08-08 10:30:35 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 205.524529ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
171.732213ms
FAIL (2025-08-08 10:29:50 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 171.764894ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
1.082679604s
FAIL (2025-08-08 10:29:44 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
6m7.755526365s) [noise=true , address=64.49.13.138:4486 , piece_id=I65IEXVW47RDLNALF3T5D2ZJKNGGH2I3MPYGJTBQM3TYWK7QJ5SA, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.240436665 (24h6m7.755539252s) [noise=true , address=104.36.82.60:4338 , piece_id=BNYKZGBWZZF7E6JSDI37L6QIU5EDD5O6PXT2ROAA7OL6TCWPJV7Q, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.240958111 (24h6m7.756060706s) [noise=true , address=64.49.13.155:4914 , piece_id=GNTXNM5KUA3A3IPU7BNG7SBDLTCDL73WVIZT4ESOB3T3K3EDFFQQ, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.241020941 (24h6m7.756123517s) [noise=true , address=104.36.82.46:4490 , piece_id=34DC6UBA55VYOYCDWBTZOSWPJSYY7LXP2A7TCGF5IDTZOLNAMMKQ, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.241505639 (24h6m7.756608224s) [noise=true , address=104.36.82.2:4751 , piece_id=CWGVZF5AN7QYDBYWXZ77S2SPAYKNVLWAIWFMOAONPES7TF2Z7ECA, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.241941247 (24h6m7.757043827s) [noise=true , address=104.36.82.23:4461 , piece_id=7XAJ3VXY4UBG3H4DFKZ3DIFUOMT755XLOCZX2VVMKX44J6B5VL7A, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.242332123 (24h6m7.757434708s) [noise=true , address=104.36.82.18:4399 , piece_id=PGFT6BNKECSMWLAZAY4UGB3T22CAXBVTCNQQIT6BPXJKF5VGYARQ, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.242753088 (24h6m7.757855664s) [noise=true , address=104.36.82.61:4407 , piece_id=R25X7USA6CPWPEIBFPT7SSDKGENAD3ZHRX3Q2KUIIX2T7JTGZ6TQ, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.242898382 (24h6m7.758000967s) [noise=true , address=104.36.82.47:4518 , piece_id=OUVHGSNAVHADJVLJDGXP25HBOINVMSLXKVWJ5VRA46L56GT5GQYQ, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.243401088 (24h6m7.758503674s) [noise=true , address=104.36.82.45:4910 , piece_id=QIY3T7A37GLYEEZQVVYYVE3J4B33H626AS7ZG6UNHG7YCW5Y5AYQ, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.243854107 (24h6m7.758956705s) [noise=true , address=104.36.82.19:4420 , piece_id=KRU3YHVKPFT3EGZ5JZDP3D5YMPMTD5Z7MWYIREFKORZ5PBXSIUMQ, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.244201456 (24h6m7.759304045s) [noise=true , address=104.36.82.58:4291 , piece_id=YRUUIYL4HOPIO7YZNTWUDJLB2QAKRGG2FZ6CNRLKWMTJXFM7Q42A, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.244608042 (24h6m7.75971062s) [noise=true , address=104.36.82.43:4845 , piece_id=PUQJ7AEUWGTLYD4MP36DLJU6QCAFEZJ6QGOG63CDOGL7R6IF44QA, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.244984412 (24h6m7.760086992s) [noise=true , address=104.36.82.59:4316 , piece_id=S6C7JBJP2IAQVNVX6N5HV6YS767T4KURXVRQMYYM5IG7ZDUXJGEA, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.245346432 (24h6m7.760449018s) [noise=true , address=104.36.82.6:4893 , piece_id=IVE3J7454ZQ25XMXLQXS7LSAQOB5HRHKQX6XSPHPTO23QEYQCIFQ, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.245779352 (24h6m7.760881931s) [noise=true , address=104.36.82.55:4223 , piece_id=THCU7GG2HXNYFOMYG2OEIN3NENSUWNHQBJACVPWVCRHWUIJPFJNA, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.246144583 (24h6m7.761247163s) [noise=true , address=104.36.82.37:4675 , piece_id=NTOV4KOJATNQC5I6ETAA2C25HDIBIFE4TEYSMDPLQCBV5YHNJBKA, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.246572369 (24h6m7.761674958s) [noise=true , address=104.36.82.9:4011 , piece_id=V4BZ2WI52UVHMRQESC6752H7R2CS3ZL7GMOEWO4MH2UEDEQOIAEA, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.364941081 (24h6m7.880043667s) [upload=24942]: Next returned nil segment
Aug 8 09:45:33.364958550 (24h6m7.880061137s) [upload=24942]: Waiting for error group managing segments...
Aug 8 09:45:33.373895698 (24h6m7.888998268s) [noise=true , address=104.36.82.59:4316 , piece_id=S6C7JBJP2IAQVNVX6N5HV6YS767T4KURXVRQMYYM5IG7ZDUXJGEA, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.375123017 (24h6m7.890225586s) [noise=true , address=104.36.82.58:4291 , piece_id=YRUUIYL4HOPIO7YZNTWUDJLB2QAKRGG2FZ6CNRLKWMTJXFM7Q42A, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.376608376 (24h6m7.891710958s) [noise=true , address=104.36.82.6:4893 , piece_id=IVE3J7454ZQ25XMXLQXS7LSAQOB5HRHKQX6XSPHPTO23QEYQCIFQ, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.377226857 (24h6m7.892329422s) [noise=true , address=104.36.82.37:4675 , piece_id=NTOV4KOJATNQC5I6ETAA2C25HDIBIFE4TEYSMDPLQCBV5YHNJBKA, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.377823264 (24h6m7.892925843s) [noise=true , address=104.36.82.2:4751 , piece_id=CWGVZF5AN7QYDBYWXZ77S2SPAYKNVLWAIWFMOAONPES7TF2Z7ECA, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.378376738 (24h6m7.893479304s) [noise=true , address=104.36.82.47:4518 , piece_id=OUVHGSNAVHADJVLJDGXP25HBOINVMSLXKVWJ5VRA46L56GT5GQYQ, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.378929934 (24h6m7.894032497s) [noise=true , address=104.36.82.23:4461 , piece_id=7XAJ3VXY4UBG3H4DFKZ3DIFUOMT755XLOCZX2VVMKX44J6B5VL7A, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.379502546 (24h6m7.894605108s) [noise=true , address=104.36.82.46:4490 , piece_id=34DC6UBA55VYOYCDWBTZOSWPJSYY7LXP2A7TCGF5IDTZOLNAMMKQ, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.379732394 (24h6m7.894834956s) [noise=true , address=104.36.82.18:4399 , piece_id=PGFT6BNKECSMWLAZAY4UGB3T22CAXBVTCNQQIT6BPXJKF5VGYARQ, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.380020859 (24h6m7.895123437s) [noise=true , address=104.36.82.60:4338 , piece_id=BNYKZGBWZZF7E6JSDI37L6QIU5EDD5O6PXT2ROAA7OL6TCWPJV7Q, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.380247285 (24h6m7.895349853s) [noise=true , address=104.36.82.9:4011 , piece_id=V4BZ2WI52UVHMRQESC6752H7R2CS3ZL7GMOEWO4MH2UEDEQOIAEA, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.380655783 (24h6m7.895758359s) [noise=true , address=104.36.82.43:4845 , piece_id=PUQJ7AEUWGTLYD4MP36DLJU6QCAFEZJ6QGOG63CDOGL7R6IF44QA, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.380663255 (24h6m7.895765826s) [noise=true , address=104.36.82.45:4910 , piece_id=QIY3T7A37GLYEEZQVVYYVE3J4B33H626AS7ZG6UNHG7YCW5Y5AYQ, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.381275036 (24h6m7.896377617s) [noise=true , address=104.36.82.61:4407 , piece_id=R25X7USA6CPWPEIBFPT7SSDKGENAD3ZHRX3Q2KUIIX2T7JTGZ6TQ, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.381311926 (24h6m7.896414503s) [seg_pos={0 0}, upload=24942]: Segment reached optimal threshold of 14 pieces.
Aug 8 09:45:33.381496675 (24h6m7.896599243s) [noise=true , address=104.36.82.55:4223 , piece_id=THCU7GG2HXNYFOMYG2OEIN3NENSUWNHQBJACVPWVCRHWUIJPFJNA, seg_pos={0 0}, upload=24942]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FijvRUiKwjk3A6yFZxNeLhPQ4bj5cxeEo6mjEs2DtRnK5WJr6): protocol: expected piece hash; context canceled; EOF
Aug 8 09:45:33.381713772 (24h6m7.896816341s) [noise=true , address=64.49.13.155:4914 , piece_id=GNTXNM5KUA3A3IPU7BNG7SBDLTCDL73WVIZT4ESOB3T3K3EDFFQQ, seg_pos={0 0}, upload=24942]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123exKWZgTdDY8ojhiFcaXkAQnsVs2MZp5MWfcaPiwk5f5tFR9A): protocol: read tcp 159.223.143.39:35526->64.49.13.155:4914: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 09:45:33.382970690 (24h6m7.898073259s) [noise=true , address=104.36.82.19:4420 , piece_id=KRU3YHVKPFT3EGZ5JZDP3D5YMPMTD5Z7MWYIREFKORZ5PBXSIUMQ, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.383418984 (24h6m7.898521552s) [noise=true , address=64.49.13.138:4486 , piece_id=I65IEXVW47RDLNALF3T5D2ZJKNGGH2I3MPYGJTBQM3TYWK7QJ5SA, seg_pos={0 0}, upload=24942]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126i47Q2ngKd79VRwsSQXZn3QTrJfmvZAQdK6yqFWXkCo5ba26g): protocol: read tcp 159.223.143.39:49138->64.49.13.138:4486: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 09:45:33.383434092 (24h6m7.898536668s) [seg_pos={0 0}, upload=24942]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil>
Aug 8 09:45:33.383521671 (24h6m7.898624248s) [upload=24942]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 09:45:33.383527482 (24h6m7.898630059s) [upload=24942]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 09:45:33.439208503 (24h6m7.954311077s) [upload=24942]: Done uploading segments...
Aug 8 09:45:33.439234033 (24h6m7.954336597s) [upload=24942]: Upload finished. err: <nil>
Aug 8 09:45:52.807400465 (24h6m27.322503034s) [upload=24946]: Starting upload
Aug 8 09:45:52.807484609 (24h6m27.322587186s) [upload=24946]: Done starting upload
Aug 8 09:45:52.807651751 (24h6m27.322754326s) [upload=24946]: Uploading segments...
Aug 8 09:45:52.808057256 (24h6m27.323159832s) [upload=24946]: Got next segment. Inline: false
Aug 8 09:45:52.808103194 (24h6m27.32320577s) [upload=24946]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 09:45:52.808108266 (24h6m27.323210845s) [upload=24946]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 09:45:52.815961170 (24h6m27.331063747s) [upload=24946]: Next segment err: metaclient: Too Many Requests
Aug 8 09:45:52.815981234 (24h6m27.331083815s) [upload=24946]: Done uploading segments...
Aug 8 09:45:52.816000021 (24h6m27.331102591s) [upload=24946]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.082709483s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
214.655318ms
FAIL (2025-08-08 10:29:25 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 8 10:16:24.996444595 (24h36m59.511547171s) [upload=25489]: Upload finished. err: <nil> Aug 8 10:16:40.086056336 (24h37m14.601158902s) [upload=25494]: Starting upload Aug 8 10:16:40.086138218 (24h37m14.601240816s) [upload=25494]: Done starting upload Aug 8 10:16:40.086246239 (24h37m14.601348806s) [upload=25494]: Uploading segments... Aug 8 10:16:40.086322136 (24h37m14.601424723s) [upload=25494]: Got next segment. Inline: true Aug 8 10:16:40.086329274 (24h37m14.601431857s) [upload=25494]: Waiting for error group managing segments... Aug 8 10:16:40.086406338 (24h37m14.601508921s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:16:40.086412709 (24h37m14.601515298s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:16:40.086433433 (24h37m14.601536015s) [upload=25494]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:16:40.147707180 (24h37m14.66280976s) [upload=25494]: Done uploading segments... Aug 8 10:16:40.147727908 (24h37m14.662830485s) [upload=25494]: Upload finished. err: <nil> Aug 8 10:17:33.679048155 (24h38m8.194150731s) [upload=25512]: Starting upload Aug 8 10:17:33.679106608 (24h38m8.194209187s) [upload=25512]: Done starting upload Aug 8 10:17:33.679160905 (24h38m8.194263481s) [upload=25512]: Uploading segments... Aug 8 10:17:33.679237763 (24h38m8.194340367s) [upload=25512]: Got next segment. Inline: true Aug 8 10:17:33.679243473 (24h38m8.194346036s) [upload=25512]: Waiting for error group managing segments... Aug 8 10:17:33.679305479 (24h38m8.194408056s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:17:33.679310678 (24h38m8.194413245s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:17:33.679313670 (24h38m8.194416232s) [upload=25512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:17:33.743363385 (24h38m8.258465953s) [upload=25512]: Done uploading segments... Aug 8 10:17:33.743382354 (24h38m8.258484924s) [upload=25512]: Upload finished. err: <nil> Aug 8 10:18:09.841643530 (24h38m44.356746109s) [upload=25524]: Starting upload Aug 8 10:18:09.841707253 (24h38m44.356809822s) [upload=25524]: Done starting upload Aug 8 10:18:09.841782697 (24h38m44.35688526s) [upload=25524]: Uploading segments... Aug 8 10:18:09.841827030 (24h38m44.356929615s) [upload=25524]: Got next segment. Inline: true Aug 8 10:18:09.841845384 (24h38m44.356947962s) [upload=25524]: Waiting for error group managing segments... Aug 8 10:18:09.841892035 (24h38m44.356994612s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:18:09.841901651 (24h38m44.357004228s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:18:09.841905159 (24h38m44.357007736s) [upload=25524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:18:09.905896485 (24h38m44.420999051s) [upload=25524]: Done uploading segments... Aug 8 10:18:09.905913027 (24h38m44.421015593s) [upload=25524]: Upload finished. err: <nil> Aug 8 10:19:23.131461453 (24h39m57.646564033s) [upload=25542]: Starting upload Aug 8 10:19:23.131505897 (24h39m57.646608483s) [upload=25542]: Done starting upload Aug 8 10:19:23.131586149 (24h39m57.646688715s) [upload=25542]: Uploading segments... Aug 8 10:19:23.131649692 (24h39m57.646752277s) [upload=25542]: Got next segment. Inline: true Aug 8 10:19:23.131655195 (24h39m57.646757771s) [upload=25542]: Waiting for error group managing segments... Aug 8 10:19:23.131714698 (24h39m57.646817274s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:23.131719532 (24h39m57.646822109s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:23.131722759 (24h39m57.646825336s) [upload=25542]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:23.206304231 (24h39m57.721406802s) [upload=25542]: Done uploading segments... Aug 8 10:19:23.206326930 (24h39m57.721429497s) [upload=25542]: Upload finished. err: <nil> Aug 8 10:19:54.392300720 (24h40m28.907403287s) [upload=25548]: Starting upload Aug 8 10:19:54.392349163 (24h40m28.907451742s) [upload=25548]: Done starting upload Aug 8 10:19:54.392419702 (24h40m28.907522272s) [upload=25548]: Uploading segments... Aug 8 10:19:54.392455430 (24h40m28.907558015s) [upload=25548]: Got next segment. Inline: true Aug 8 10:19:54.392460240 (24h40m28.907562806s) [upload=25548]: Waiting for error group managing segments... Aug 8 10:19:54.392508955 (24h40m28.907611521s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:19:54.392513704 (24h40m28.907616283s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:19:54.392516750 (24h40m28.907619315s) [upload=25548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:19:54.607175596 (24h40m29.122278175s) [upload=25548]: Done uploading segments... Aug 8 10:19:54.607193769 (24h40m29.122296335s) [upload=25548]: Upload finished. err: <nil> Aug 8 10:21:05.692737556 (24h41m40.207840123s) [upload=25567]: Starting upload Aug 8 10:21:05.692815089 (24h41m40.207917657s) [upload=25567]: Done starting upload Aug 8 10:21:05.692894948 (24h41m40.207997514s) [upload=25567]: Uploading segments... Aug 8 10:21:05.692944055 (24h41m40.208046641s) [upload=25567]: Got next segment. Inline: true Aug 8 10:21:05.692950912 (24h41m40.208053489s) [upload=25567]: Waiting for error group managing segments... Aug 8 10:21:05.692998236 (24h41m40.208100823s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:05.693004179 (24h41m40.20810676s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:05.693007287 (24h41m40.208109867s) [upload=25567]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:05.757359800 (24h41m40.272462386s) [upload=25567]: Done uploading segments... Aug 8 10:21:05.757383050 (24h41m40.272485626s) [upload=25567]: Upload finished. err: <nil> Aug 8 10:21:53.504743996 (24h42m28.019846582s) [upload=25582]: Starting upload Aug 8 10:21:53.504817681 (24h42m28.019920261s) [upload=25582]: Done starting upload Aug 8 10:21:53.504908346 (24h42m28.020010928s) [upload=25582]: Uploading segments... Aug 8 10:21:53.504956252 (24h42m28.020058836s) [upload=25582]: Got next segment. Inline: true Aug 8 10:21:53.504962939 (24h42m28.020065516s) [upload=25582]: Waiting for error group managing segments... Aug 8 10:21:53.505014143 (24h42m28.020116729s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:21:53.505020195 (24h42m28.020122779s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:21:53.505024975 (24h42m28.020127552s) [upload=25582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:21:53.567859466 (24h42m28.082962043s) [upload=25582]: Done uploading segments... Aug 8 10:21:53.567875292 (24h42m28.082977868s) [upload=25582]: Upload finished. err: <nil> Aug 8 10:22:57.956939547 (24h43m32.472042133s) [upload=25598]: Starting upload Aug 8 10:22:57.956995820 (24h43m32.472098408s) [upload=25598]: Done starting upload Aug 8 10:22:57.957240080 (24h43m32.472342648s) [upload=25598]: Uploading segments... Aug 8 10:22:57.957311205 (24h43m32.472413791s) [upload=25598]: Got next segment. Inline: true Aug 8 10:22:57.957317746 (24h43m32.472420323s) [upload=25598]: Waiting for error group managing segments... Aug 8 10:22:57.957385033 (24h43m32.472487614s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:22:57.957390668 (24h43m32.472493246s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:22:57.957394180 (24h43m32.472496757s) [upload=25598]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:22:58.029812590 (24h43m32.544915169s) [upload=25598]: Done uploading segments... Aug 8 10:22:58.029834230 (24h43m32.544936798s) [upload=25598]: Upload finished. err: <nil> Aug 8 10:23:18.816059986 (24h43m53.331162571s) [upload=25604]: Starting upload Aug 8 10:23:18.816111932 (24h43m53.331214508s) [upload=25604]: Done starting upload Aug 8 10:23:18.816172180 (24h43m53.331274751s) [upload=25604]: Uploading segments... Aug 8 10:23:18.816210063 (24h43m53.331312649s) [upload=25604]: Got next segment. Inline: true Aug 8 10:23:18.816214426 (24h43m53.331316991s) [upload=25604]: Waiting for error group managing segments... Aug 8 10:23:18.816251408 (24h43m53.331353974s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:23:18.816255697 (24h43m53.331358273s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:23:18.816258801 (24h43m53.331361371s) [upload=25604]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:23:19.087130277 (24h43m53.602232846s) [upload=25604]: Done uploading segments... Aug 8 10:23:19.087151214 (24h43m53.602253781s) [upload=25604]: Upload finished. err: <nil> Aug 8 10:24:21.030100948 (24h44m55.545203519s) [upload=25623]: Starting upload Aug 8 10:24:21.030162897 (24h44m55.545265478s) [upload=25623]: Done starting upload Aug 8 10:24:21.030247890 (24h44m55.545350466s) [upload=25623]: Uploading segments... Aug 8 10:24:21.030299798 (24h44m55.545402387s) [upload=25623]: Got next segment. Inline: true Aug 8 10:24:21.030306830 (24h44m55.545409394s) [upload=25623]: Waiting for error group managing segments... Aug 8 10:24:21.030431847 (24h44m55.545534414s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 10:24:21.030438082 (24h44m55.545540659s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 10:24:21.030441385 (24h44m55.545543966s) [upload=25623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 10:24:21.039318112 (24h44m55.554420681s) [upload=25623]: Done uploading segments... Aug 8 10:24:21.039334812 (24h44m55.554437379s) [upload=25623]: Upload finished. err: metaclient: Too Many Requests
Duration: 214.70479ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
761.467255ms
FAIL (2025-08-08 10:29:09 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
6m7.755526365s) [noise=true , address=64.49.13.138:4486 , piece_id=I65IEXVW47RDLNALF3T5D2ZJKNGGH2I3MPYGJTBQM3TYWK7QJ5SA, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.240436665 (24h6m7.755539252s) [noise=true , address=104.36.82.60:4338 , piece_id=BNYKZGBWZZF7E6JSDI37L6QIU5EDD5O6PXT2ROAA7OL6TCWPJV7Q, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.240958111 (24h6m7.756060706s) [noise=true , address=64.49.13.155:4914 , piece_id=GNTXNM5KUA3A3IPU7BNG7SBDLTCDL73WVIZT4ESOB3T3K3EDFFQQ, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.241020941 (24h6m7.756123517s) [noise=true , address=104.36.82.46:4490 , piece_id=34DC6UBA55VYOYCDWBTZOSWPJSYY7LXP2A7TCGF5IDTZOLNAMMKQ, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.241505639 (24h6m7.756608224s) [noise=true , address=104.36.82.2:4751 , piece_id=CWGVZF5AN7QYDBYWXZ77S2SPAYKNVLWAIWFMOAONPES7TF2Z7ECA, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.241941247 (24h6m7.757043827s) [noise=true , address=104.36.82.23:4461 , piece_id=7XAJ3VXY4UBG3H4DFKZ3DIFUOMT755XLOCZX2VVMKX44J6B5VL7A, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.242332123 (24h6m7.757434708s) [noise=true , address=104.36.82.18:4399 , piece_id=PGFT6BNKECSMWLAZAY4UGB3T22CAXBVTCNQQIT6BPXJKF5VGYARQ, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.242753088 (24h6m7.757855664s) [noise=true , address=104.36.82.61:4407 , piece_id=R25X7USA6CPWPEIBFPT7SSDKGENAD3ZHRX3Q2KUIIX2T7JTGZ6TQ, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.242898382 (24h6m7.758000967s) [noise=true , address=104.36.82.47:4518 , piece_id=OUVHGSNAVHADJVLJDGXP25HBOINVMSLXKVWJ5VRA46L56GT5GQYQ, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.243401088 (24h6m7.758503674s) [noise=true , address=104.36.82.45:4910 , piece_id=QIY3T7A37GLYEEZQVVYYVE3J4B33H626AS7ZG6UNHG7YCW5Y5AYQ, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.243854107 (24h6m7.758956705s) [noise=true , address=104.36.82.19:4420 , piece_id=KRU3YHVKPFT3EGZ5JZDP3D5YMPMTD5Z7MWYIREFKORZ5PBXSIUMQ, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.244201456 (24h6m7.759304045s) [noise=true , address=104.36.82.58:4291 , piece_id=YRUUIYL4HOPIO7YZNTWUDJLB2QAKRGG2FZ6CNRLKWMTJXFM7Q42A, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.244608042 (24h6m7.75971062s) [noise=true , address=104.36.82.43:4845 , piece_id=PUQJ7AEUWGTLYD4MP36DLJU6QCAFEZJ6QGOG63CDOGL7R6IF44QA, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.244984412 (24h6m7.760086992s) [noise=true , address=104.36.82.59:4316 , piece_id=S6C7JBJP2IAQVNVX6N5HV6YS767T4KURXVRQMYYM5IG7ZDUXJGEA, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.245346432 (24h6m7.760449018s) [noise=true , address=104.36.82.6:4893 , piece_id=IVE3J7454ZQ25XMXLQXS7LSAQOB5HRHKQX6XSPHPTO23QEYQCIFQ, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.245779352 (24h6m7.760881931s) [noise=true , address=104.36.82.55:4223 , piece_id=THCU7GG2HXNYFOMYG2OEIN3NENSUWNHQBJACVPWVCRHWUIJPFJNA, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.246144583 (24h6m7.761247163s) [noise=true , address=104.36.82.37:4675 , piece_id=NTOV4KOJATNQC5I6ETAA2C25HDIBIFE4TEYSMDPLQCBV5YHNJBKA, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.246572369 (24h6m7.761674958s) [noise=true , address=104.36.82.9:4011 , piece_id=V4BZ2WI52UVHMRQESC6752H7R2CS3ZL7GMOEWO4MH2UEDEQOIAEA, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.364941081 (24h6m7.880043667s) [upload=24942]: Next returned nil segment
Aug 8 09:45:33.364958550 (24h6m7.880061137s) [upload=24942]: Waiting for error group managing segments...
Aug 8 09:45:33.373895698 (24h6m7.888998268s) [noise=true , address=104.36.82.59:4316 , piece_id=S6C7JBJP2IAQVNVX6N5HV6YS767T4KURXVRQMYYM5IG7ZDUXJGEA, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.375123017 (24h6m7.890225586s) [noise=true , address=104.36.82.58:4291 , piece_id=YRUUIYL4HOPIO7YZNTWUDJLB2QAKRGG2FZ6CNRLKWMTJXFM7Q42A, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.376608376 (24h6m7.891710958s) [noise=true , address=104.36.82.6:4893 , piece_id=IVE3J7454ZQ25XMXLQXS7LSAQOB5HRHKQX6XSPHPTO23QEYQCIFQ, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.377226857 (24h6m7.892329422s) [noise=true , address=104.36.82.37:4675 , piece_id=NTOV4KOJATNQC5I6ETAA2C25HDIBIFE4TEYSMDPLQCBV5YHNJBKA, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.377823264 (24h6m7.892925843s) [noise=true , address=104.36.82.2:4751 , piece_id=CWGVZF5AN7QYDBYWXZ77S2SPAYKNVLWAIWFMOAONPES7TF2Z7ECA, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.378376738 (24h6m7.893479304s) [noise=true , address=104.36.82.47:4518 , piece_id=OUVHGSNAVHADJVLJDGXP25HBOINVMSLXKVWJ5VRA46L56GT5GQYQ, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.378929934 (24h6m7.894032497s) [noise=true , address=104.36.82.23:4461 , piece_id=7XAJ3VXY4UBG3H4DFKZ3DIFUOMT755XLOCZX2VVMKX44J6B5VL7A, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.379502546 (24h6m7.894605108s) [noise=true , address=104.36.82.46:4490 , piece_id=34DC6UBA55VYOYCDWBTZOSWPJSYY7LXP2A7TCGF5IDTZOLNAMMKQ, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.379732394 (24h6m7.894834956s) [noise=true , address=104.36.82.18:4399 , piece_id=PGFT6BNKECSMWLAZAY4UGB3T22CAXBVTCNQQIT6BPXJKF5VGYARQ, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.380020859 (24h6m7.895123437s) [noise=true , address=104.36.82.60:4338 , piece_id=BNYKZGBWZZF7E6JSDI37L6QIU5EDD5O6PXT2ROAA7OL6TCWPJV7Q, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.380247285 (24h6m7.895349853s) [noise=true , address=104.36.82.9:4011 , piece_id=V4BZ2WI52UVHMRQESC6752H7R2CS3ZL7GMOEWO4MH2UEDEQOIAEA, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.380655783 (24h6m7.895758359s) [noise=true , address=104.36.82.43:4845 , piece_id=PUQJ7AEUWGTLYD4MP36DLJU6QCAFEZJ6QGOG63CDOGL7R6IF44QA, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.380663255 (24h6m7.895765826s) [noise=true , address=104.36.82.45:4910 , piece_id=QIY3T7A37GLYEEZQVVYYVE3J4B33H626AS7ZG6UNHG7YCW5Y5AYQ, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.381275036 (24h6m7.896377617s) [noise=true , address=104.36.82.61:4407 , piece_id=R25X7USA6CPWPEIBFPT7SSDKGENAD3ZHRX3Q2KUIIX2T7JTGZ6TQ, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.381311926 (24h6m7.896414503s) [seg_pos={0 0}, upload=24942]: Segment reached optimal threshold of 14 pieces.
Aug 8 09:45:33.381496675 (24h6m7.896599243s) [noise=true , address=104.36.82.55:4223 , piece_id=THCU7GG2HXNYFOMYG2OEIN3NENSUWNHQBJACVPWVCRHWUIJPFJNA, seg_pos={0 0}, upload=24942]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FijvRUiKwjk3A6yFZxNeLhPQ4bj5cxeEo6mjEs2DtRnK5WJr6): protocol: expected piece hash; context canceled; EOF
Aug 8 09:45:33.381713772 (24h6m7.896816341s) [noise=true , address=64.49.13.155:4914 , piece_id=GNTXNM5KUA3A3IPU7BNG7SBDLTCDL73WVIZT4ESOB3T3K3EDFFQQ, seg_pos={0 0}, upload=24942]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123exKWZgTdDY8ojhiFcaXkAQnsVs2MZp5MWfcaPiwk5f5tFR9A): protocol: read tcp 159.223.143.39:35526->64.49.13.155:4914: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 09:45:33.382970690 (24h6m7.898073259s) [noise=true , address=104.36.82.19:4420 , piece_id=KRU3YHVKPFT3EGZ5JZDP3D5YMPMTD5Z7MWYIREFKORZ5PBXSIUMQ, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.383418984 (24h6m7.898521552s) [noise=true , address=64.49.13.138:4486 , piece_id=I65IEXVW47RDLNALF3T5D2ZJKNGGH2I3MPYGJTBQM3TYWK7QJ5SA, seg_pos={0 0}, upload=24942]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126i47Q2ngKd79VRwsSQXZn3QTrJfmvZAQdK6yqFWXkCo5ba26g): protocol: read tcp 159.223.143.39:49138->64.49.13.138:4486: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 09:45:33.383434092 (24h6m7.898536668s) [seg_pos={0 0}, upload=24942]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil>
Aug 8 09:45:33.383521671 (24h6m7.898624248s) [upload=24942]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 09:45:33.383527482 (24h6m7.898630059s) [upload=24942]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 09:45:33.439208503 (24h6m7.954311077s) [upload=24942]: Done uploading segments...
Aug 8 09:45:33.439234033 (24h6m7.954336597s) [upload=24942]: Upload finished. err: <nil>
Aug 8 09:45:52.807400465 (24h6m27.322503034s) [upload=24946]: Starting upload
Aug 8 09:45:52.807484609 (24h6m27.322587186s) [upload=24946]: Done starting upload
Aug 8 09:45:52.807651751 (24h6m27.322754326s) [upload=24946]: Uploading segments...
Aug 8 09:45:52.808057256 (24h6m27.323159832s) [upload=24946]: Got next segment. Inline: false
Aug 8 09:45:52.808103194 (24h6m27.32320577s) [upload=24946]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 09:45:52.808108266 (24h6m27.323210845s) [upload=24946]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 09:45:52.815961170 (24h6m27.331063747s) [upload=24946]: Next segment err: metaclient: Too Many Requests
Aug 8 09:45:52.815981234 (24h6m27.331083815s) [upload=24946]: Done uploading segments...
Aug 8 09:45:52.816000021 (24h6m27.331102591s) [upload=24946]: Upload finished. err: metaclient: Too Many Requests
Duration: 761.497454ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
379.792079ms
FAIL (2025-08-08 10:28:52 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
6m7.755526365s) [noise=true , address=64.49.13.138:4486 , piece_id=I65IEXVW47RDLNALF3T5D2ZJKNGGH2I3MPYGJTBQM3TYWK7QJ5SA, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.240436665 (24h6m7.755539252s) [noise=true , address=104.36.82.60:4338 , piece_id=BNYKZGBWZZF7E6JSDI37L6QIU5EDD5O6PXT2ROAA7OL6TCWPJV7Q, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.240958111 (24h6m7.756060706s) [noise=true , address=64.49.13.155:4914 , piece_id=GNTXNM5KUA3A3IPU7BNG7SBDLTCDL73WVIZT4ESOB3T3K3EDFFQQ, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.241020941 (24h6m7.756123517s) [noise=true , address=104.36.82.46:4490 , piece_id=34DC6UBA55VYOYCDWBTZOSWPJSYY7LXP2A7TCGF5IDTZOLNAMMKQ, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.241505639 (24h6m7.756608224s) [noise=true , address=104.36.82.2:4751 , piece_id=CWGVZF5AN7QYDBYWXZ77S2SPAYKNVLWAIWFMOAONPES7TF2Z7ECA, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.241941247 (24h6m7.757043827s) [noise=true , address=104.36.82.23:4461 , piece_id=7XAJ3VXY4UBG3H4DFKZ3DIFUOMT755XLOCZX2VVMKX44J6B5VL7A, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.242332123 (24h6m7.757434708s) [noise=true , address=104.36.82.18:4399 , piece_id=PGFT6BNKECSMWLAZAY4UGB3T22CAXBVTCNQQIT6BPXJKF5VGYARQ, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.242753088 (24h6m7.757855664s) [noise=true , address=104.36.82.61:4407 , piece_id=R25X7USA6CPWPEIBFPT7SSDKGENAD3ZHRX3Q2KUIIX2T7JTGZ6TQ, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.242898382 (24h6m7.758000967s) [noise=true , address=104.36.82.47:4518 , piece_id=OUVHGSNAVHADJVLJDGXP25HBOINVMSLXKVWJ5VRA46L56GT5GQYQ, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.243401088 (24h6m7.758503674s) [noise=true , address=104.36.82.45:4910 , piece_id=QIY3T7A37GLYEEZQVVYYVE3J4B33H626AS7ZG6UNHG7YCW5Y5AYQ, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.243854107 (24h6m7.758956705s) [noise=true , address=104.36.82.19:4420 , piece_id=KRU3YHVKPFT3EGZ5JZDP3D5YMPMTD5Z7MWYIREFKORZ5PBXSIUMQ, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.244201456 (24h6m7.759304045s) [noise=true , address=104.36.82.58:4291 , piece_id=YRUUIYL4HOPIO7YZNTWUDJLB2QAKRGG2FZ6CNRLKWMTJXFM7Q42A, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.244608042 (24h6m7.75971062s) [noise=true , address=104.36.82.43:4845 , piece_id=PUQJ7AEUWGTLYD4MP36DLJU6QCAFEZJ6QGOG63CDOGL7R6IF44QA, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.244984412 (24h6m7.760086992s) [noise=true , address=104.36.82.59:4316 , piece_id=S6C7JBJP2IAQVNVX6N5HV6YS767T4KURXVRQMYYM5IG7ZDUXJGEA, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.245346432 (24h6m7.760449018s) [noise=true , address=104.36.82.6:4893 , piece_id=IVE3J7454ZQ25XMXLQXS7LSAQOB5HRHKQX6XSPHPTO23QEYQCIFQ, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.245779352 (24h6m7.760881931s) [noise=true , address=104.36.82.55:4223 , piece_id=THCU7GG2HXNYFOMYG2OEIN3NENSUWNHQBJACVPWVCRHWUIJPFJNA, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.246144583 (24h6m7.761247163s) [noise=true , address=104.36.82.37:4675 , piece_id=NTOV4KOJATNQC5I6ETAA2C25HDIBIFE4TEYSMDPLQCBV5YHNJBKA, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.246572369 (24h6m7.761674958s) [noise=true , address=104.36.82.9:4011 , piece_id=V4BZ2WI52UVHMRQESC6752H7R2CS3ZL7GMOEWO4MH2UEDEQOIAEA, seg_pos={0 0}, upload=24942]: Uploading piece...
Aug 8 09:45:33.364941081 (24h6m7.880043667s) [upload=24942]: Next returned nil segment
Aug 8 09:45:33.364958550 (24h6m7.880061137s) [upload=24942]: Waiting for error group managing segments...
Aug 8 09:45:33.373895698 (24h6m7.888998268s) [noise=true , address=104.36.82.59:4316 , piece_id=S6C7JBJP2IAQVNVX6N5HV6YS767T4KURXVRQMYYM5IG7ZDUXJGEA, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.375123017 (24h6m7.890225586s) [noise=true , address=104.36.82.58:4291 , piece_id=YRUUIYL4HOPIO7YZNTWUDJLB2QAKRGG2FZ6CNRLKWMTJXFM7Q42A, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.376608376 (24h6m7.891710958s) [noise=true , address=104.36.82.6:4893 , piece_id=IVE3J7454ZQ25XMXLQXS7LSAQOB5HRHKQX6XSPHPTO23QEYQCIFQ, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.377226857 (24h6m7.892329422s) [noise=true , address=104.36.82.37:4675 , piece_id=NTOV4KOJATNQC5I6ETAA2C25HDIBIFE4TEYSMDPLQCBV5YHNJBKA, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.377823264 (24h6m7.892925843s) [noise=true , address=104.36.82.2:4751 , piece_id=CWGVZF5AN7QYDBYWXZ77S2SPAYKNVLWAIWFMOAONPES7TF2Z7ECA, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.378376738 (24h6m7.893479304s) [noise=true , address=104.36.82.47:4518 , piece_id=OUVHGSNAVHADJVLJDGXP25HBOINVMSLXKVWJ5VRA46L56GT5GQYQ, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.378929934 (24h6m7.894032497s) [noise=true , address=104.36.82.23:4461 , piece_id=7XAJ3VXY4UBG3H4DFKZ3DIFUOMT755XLOCZX2VVMKX44J6B5VL7A, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.379502546 (24h6m7.894605108s) [noise=true , address=104.36.82.46:4490 , piece_id=34DC6UBA55VYOYCDWBTZOSWPJSYY7LXP2A7TCGF5IDTZOLNAMMKQ, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.379732394 (24h6m7.894834956s) [noise=true , address=104.36.82.18:4399 , piece_id=PGFT6BNKECSMWLAZAY4UGB3T22CAXBVTCNQQIT6BPXJKF5VGYARQ, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.380020859 (24h6m7.895123437s) [noise=true , address=104.36.82.60:4338 , piece_id=BNYKZGBWZZF7E6JSDI37L6QIU5EDD5O6PXT2ROAA7OL6TCWPJV7Q, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.380247285 (24h6m7.895349853s) [noise=true , address=104.36.82.9:4011 , piece_id=V4BZ2WI52UVHMRQESC6752H7R2CS3ZL7GMOEWO4MH2UEDEQOIAEA, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.380655783 (24h6m7.895758359s) [noise=true , address=104.36.82.43:4845 , piece_id=PUQJ7AEUWGTLYD4MP36DLJU6QCAFEZJ6QGOG63CDOGL7R6IF44QA, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.380663255 (24h6m7.895765826s) [noise=true , address=104.36.82.45:4910 , piece_id=QIY3T7A37GLYEEZQVVYYVE3J4B33H626AS7ZG6UNHG7YCW5Y5AYQ, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.381275036 (24h6m7.896377617s) [noise=true , address=104.36.82.61:4407 , piece_id=R25X7USA6CPWPEIBFPT7SSDKGENAD3ZHRX3Q2KUIIX2T7JTGZ6TQ, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.381311926 (24h6m7.896414503s) [seg_pos={0 0}, upload=24942]: Segment reached optimal threshold of 14 pieces.
Aug 8 09:45:33.381496675 (24h6m7.896599243s) [noise=true , address=104.36.82.55:4223 , piece_id=THCU7GG2HXNYFOMYG2OEIN3NENSUWNHQBJACVPWVCRHWUIJPFJNA, seg_pos={0 0}, upload=24942]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FijvRUiKwjk3A6yFZxNeLhPQ4bj5cxeEo6mjEs2DtRnK5WJr6): protocol: expected piece hash; context canceled; EOF
Aug 8 09:45:33.381713772 (24h6m7.896816341s) [noise=true , address=64.49.13.155:4914 , piece_id=GNTXNM5KUA3A3IPU7BNG7SBDLTCDL73WVIZT4ESOB3T3K3EDFFQQ, seg_pos={0 0}, upload=24942]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123exKWZgTdDY8ojhiFcaXkAQnsVs2MZp5MWfcaPiwk5f5tFR9A): protocol: read tcp 159.223.143.39:35526->64.49.13.155:4914: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 09:45:33.382970690 (24h6m7.898073259s) [noise=true , address=104.36.82.19:4420 , piece_id=KRU3YHVKPFT3EGZ5JZDP3D5YMPMTD5Z7MWYIREFKORZ5PBXSIUMQ, seg_pos={0 0}, upload=24942]: Done uploading piece. err: <nil>
Aug 8 09:45:33.383418984 (24h6m7.898521552s) [noise=true , address=64.49.13.138:4486 , piece_id=I65IEXVW47RDLNALF3T5D2ZJKNGGH2I3MPYGJTBQM3TYWK7QJ5SA, seg_pos={0 0}, upload=24942]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126i47Q2ngKd79VRwsSQXZn3QTrJfmvZAQdK6yqFWXkCo5ba26g): protocol: read tcp 159.223.143.39:49138->64.49.13.138:4486: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 09:45:33.383434092 (24h6m7.898536668s) [seg_pos={0 0}, upload=24942]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil>
Aug 8 09:45:33.383521671 (24h6m7.898624248s) [upload=24942]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 09:45:33.383527482 (24h6m7.898630059s) [upload=24942]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 09:45:33.439208503 (24h6m7.954311077s) [upload=24942]: Done uploading segments...
Aug 8 09:45:33.439234033 (24h6m7.954336597s) [upload=24942]: Upload finished. err: <nil>
Aug 8 09:45:52.807400465 (24h6m27.322503034s) [upload=24946]: Starting upload
Aug 8 09:45:52.807484609 (24h6m27.322587186s) [upload=24946]: Done starting upload
Aug 8 09:45:52.807651751 (24h6m27.322754326s) [upload=24946]: Uploading segments...
Aug 8 09:45:52.808057256 (24h6m27.323159832s) [upload=24946]: Got next segment. Inline: false
Aug 8 09:45:52.808103194 (24h6m27.32320577s) [upload=24946]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 09:45:52.808108266 (24h6m27.323210845s) [upload=24946]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 09:45:52.815961170 (24h6m27.331063747s) [upload=24946]: Next segment err: metaclient: Too Many Requests
Aug 8 09:45:52.815981234 (24h6m27.331083815s) [upload=24946]: Done uploading segments...
Aug 8 09:45:52.816000021 (24h6m27.331102591s) [upload=24946]: Upload finished. err: metaclient: Too Many Requests
Duration: 379.813638ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|