Keep at it and you'll figure it out. It's probably best not to battle other human coaches your first go round. Pick out a few quality "undecided" recruits within your 180 (recruits within 180 miles of your school cost the least to recruit). Usually 3 head coach scouts and 3 assistant coach scouts will get an undecided to consider you - as long as another human coach doesn't go after the same kid. Once a recruit is "green" for you, you don't have to spend any more money on him until you offer a scholarship (make sure you save the $100 for a scholarship offer).
You should also start learning how to battle - against Sim AI first. Pick out a recruit or two (inside your 180) that is already considering schools coached by Sim AI. Use HC and AC scouts to get on his "considering" list, then experiment with home visits and campus visits until you knock the Sims off. You'll start to figure out how much recruiting effort it takes to beat a Sim (it can vary a little, but it's usually pretty consistent) on a recruit. One piece of advice: the longer you take to knock off a Sim, the more it'll cost you.
Once you've got down beating sims, then you can try a human - but they're far less predictable and almost always more expensive to beat than a sim. Pick your battles. If you get caught up in too many, your cash will get drained fast.