I have tried using moving averages, etc, but they don't take into account a player improving.
The trick is to be aware of what your formula takes into account and what it doesn't -- and then make adjustments by hand to account for the stuff that your formula doesn't consider.A
very simple formula would be: Performance in Year N = Performance in Year N-1.
This takes into account
nothing other than last year's stats. That's okay, but it means you will have to make a
lot of manual adjustments for each player.
Another formula is to start with last year's stats, and then regress them toward the mean by an amount you determine to be reasonable based on data from previous years. You can do this for each stat individually rather than for fantasy points. Some stats, like TDs, will be regressed a lot more than others, like yards.
Getting slightly more complicated, you could use regression analysis to do projections in one statistical category based on a player's performance the previous year in many different statistical categories. For example, let's say we want to project a player's rushing TDs for this year. It might turn out that his rushing yards from last year will be even more important than his rushing TDs from last year, for purposes of making this projection, but his yards per reception, his team's winning percentage, and a bunch of other stats will be useful as well. Regression analysis can figure all this stuff out for you and give you a formula that you would have never come up with on your own.
You can keep adding in other variables like age, changes to the offensive line, and so on. Theoretically, you could use a program that takes every bit of available information into account and then spits out a projection that you cannot improve on by making any manual adjustments to it.
But that's just theoretical

stuff. As a practical matter, you will still have to use your own head and your own subjective judgment quite a bit.
Regardless of everything I wrote after my first paragraph in this post, I actually think a perfectly good "formula" is just to use last year's stats as a starting point.
Then look at a player's previous history to put last year's stats in context. If a guy rushed for 5.2 yards per carry last year, that seems pretty high, and is likely to come down next year just because of normal regression to the mean. (Regression to the mean deserves at least a few paragraphs of its own. I won't get into it here, but it is not magic or voodoo. It is a normal phenomenon that can be explained mathematically.) So if all you know is that he rushed for 5.2 ypc last year, you might expect him to rush for something like 4.7 next year. But then if you check the guy's previous history and notice that he rushed for over 5.0 ypc in each of his last four seasons, that 5.2 doesn't look like an aberration anymore. It seems like a reasonable prediction for the following year as well.
After you look at what a player did last year and put it into the context of what he's done before that in his career, you can start considering things like his age, his health, and so on.
Is he expected to improve this year, or is he on the downside of his career?
Will his role on the team change this year? Is a new offensive coordinator going to use him differently? Did the team draft a hotshot young player at the same position who will challenge him for playing time?
Did the OL get better or worse?
Will this year's schedule be tougher or easier than last year's?
Etc., etc. Basically, try to consider as many different factors as you can think of, and try to figure out how they'll all fit together to affect how next season will be different from last season.
When you're done, not only will you have a final projection for him that you can reduce down to a three-digit number (e.g., 285 projected fantasy points). But much more importantly, you will have thought about the player's skill level, his role on the team, changes to his coaching and surrounding personnel, and a bunch of other factors that will give you a solid gut feel about his prospects for next season. During the draft, that gut feel may be more important than the three-digit number since it will include info about a player's risk level and other stuff that is not reflected in the final projection but that may nonetheless be important to evaluating how he'll fit in on your fantasy team.