Is paleo like an even nerdier version of atkins?
Paleo is like Atkins except it's not low-carb, and it avoids all processed foods. They have in common that they generally avoid grains.
So is that the shtick, that you basically only eat meats or things you can pick off a tree, like early man?
Pretty much. The idea is that novel foods introduced to the human diet within the last 10,000 years or so (e.g., wheat, dairy) may be good for some people and problematic for other people, but any food that had been a regular part of the human diet for the few hundred thousand or so years before that should be non-problematic for just about everybody. So to play it safe, stick to those.
There are plenty of problems with putting the theory into practice. There's really no such thing, for example, as a generic pre-agrarian human diet. Different humans ate different diets at different times and places, and they generally ate foods that aren't available to modern Americans. But even if pigs and chickens and strawberries and spinach and sweet potatoes didn't exist during the paleolithic, they are closer to what existed than Ding Dongs or Cheetos or Pop Tarts or anything else that has to be capitalized because it's a brand name.
The paleo diet is one of many examples of a more general principle that if you want to eat healthy, find a culture that is known for its health and try to eat what they ate. That's the basis for the Mediterranean diet, the Okinawan diet, the Weston A. Price diet, and others. Our paleolithic ancestors were generally healthier than their agrarian descendants, so the theory goes that we should eat like hunter-gatherers rather than like farmers. It's rather extreme, because while farmers of 5,000 years ago were very unhealthy (as measured by life expectancy, etc.), farmers of 100 years ago were quite robust. So why not look just a hundred years back instead of a few hundred thousand? (Especially since we actually know what people ate 100 years ago, while we're still largely guessing about the paleolithic era.) There's an answer to that question, and a rebuttal to that answer, but this tangent is long enough.
The paleo diet works for weight loss, at least while people stick to it, probably because it avoids commercially prepared processed foods. The big food companies hire really smart scientists to figure out ways to make their foods more convenient, more palatable, and more addicting, which induces overeating. They process foods by reducing the fiber and water content (fiber and water both promote satiety), adding sugar and fat (increasing caloric density), and engineering the flavor to be particularly binge-worthy. It's much easier to binge on Doritos than on plain, boiled potatoes.
Atkins also avoids most junk foods. The advantage of the paleo diet over Atkins, IMO, is that Atkins seems more appropriate as a short-term (maybe a few years) diet for many people rather than a permanent way of eating. Some people may do perfectly fine on low-carb diets in the long run, but others may develop thyroid problems, elevated LDL, impaired gut flora, occasional scurvy... I don't know of similar issues inherent to long-term paleo (although there are many different versions of paleo, some of which may have similar problems).
[i'm of the opinion that paleo is overkill. While grains, dairy, and legumes may give some people problems, it's not that hard to figure out whether you're lactose intolerant and to avoid dairy if that's the case. For other people, those and other neolithic foods can be perfectly good foods. In my view, it's the foods introduced 50-100 years ago that are the problem, not the foods introduced 10,000 years ago. Nonetheless, overrestrictiveness isn't necessarily a problem unless it inhibits compliance; and for many people, I think the opposite may be true. The "taboo" nature of certain categories of foods may help compliance because taboos are good motivators.]