The Shadowmarks book tells you what all of the marks mean: protected, good loot, fence, danger, etc. The marks themselves are somewhere on the exterior of the house, generally near the door. Sometimes at ground level, sometimes above the door... somebody who wasn't looking for them would probably never notice them, but when you know what to look for, they stand out pretty well.
But not every building has one, so don't drive yourself nuts. Many residences do, though. Especially swank ones.
If you want to see an example, go to Windhelm. Spot the "protected" mark on the Clan Cruel-Sea house, and the "loot" mark on both Clan Shatter-Shield and Viola Giordano's House.