I don't do criminal law, but I've read a few things about the exclusionary rule on the Internet.
It's true that consent to a search (or, in this case, consent to simply being on the property and acting like a normal houseguest) doesn't automatically include consent to seize any evidence found during the search. Whether a warrant is required to seize such evidence will depend on the totality of the circumstances -- how easy is it for the property to be destroyed while waiting for a warrant, etc. I don't think the answer is completely clear cut in this case.
I also think the plain view exception is relevant. A warrant is not needed to seize evidence in plain view. For an item to be considered "in plain view," an officer can't move stuff around during a search to reveal it. But in this case, magazines were moved around and the book was revealed not when an officer was conducting a search, but when a houseguest was taking a dump. Once the book was revealed -- incident to a dump, not to a search -- it was then in plain view. If the houseguest had been a non-cop rather than an off-duty cop, and he inadvertently revealed the evidence in front of a cop so that it was then in plain view, I think the cop could seize it without a warrant (provided that the cop was on the premises legally -- e.g., with consent). The water is muddied here because the houseguest and the cop are the same person; but I wouldn't give up that fight if I were the prosecutor.
I don't think Hank took Leaves of Grass with the idea that it would be admissible. Even if it's admissible, it's pretty weak as evidence that Walt is Heisenberg. So I don't think its admissibility matters very much. But as for whether it really would be admissible, I don't think it's completely black and white. I think it's an interesting enough question that it could appear on a crim pro exam.