If you go out the back exit, you'll be on top of the mountain. Directly below is a river. Use the "Become Ethereal" shout and jump off the mountain to the river below. There you'll find a broken boat half moored on a thin sliver of coast. Inside is a crapload of precious gems: rubies, diamonds, emeralds, etc. In the river (underwater) is a chest with further loot.
I would have never found this except I like base jumping, so saved then used the ethereal shout and jumped. Imagine my surprise when I landed on top of the mother lode.