I give BB LOTS of credit, what he has done with the NE franchise is very impressive. However, the eye test says that many of those D's were not very good despite the Top 10 in points per game you referenced. Stats often lie.
Okay, but if we look at the losses in the Belichick/Brady era, there is only one you can put solely on the defense, the 2006 loss to the Colts in the AFCCG.
2005: 27-13 to Denver (offense was terrible, and 7 of the Broncos points came from a 1-yard drive following the Champ Bailey return of the Brady INT)
2006: 38-34 to Indy (this one is on the D)
2007: 17-14 to NY Giants (offense scored 23 points under their season average; despite the defense giving up the late score, this one is all on the offense)
2009: 33-14 to Baltimore (the whole team was not good this day, but the offense was worse, committing a series of turnovers early that allowed the Ravens to build a 24-0 1st quarter lead, that admittedly was started by a long Ray Rice TD; otherwise, the D was good that day)
2010: 28-21 to NY Jets (final score is misleading, as the Patriots D played relatively well and the O did not, but the Jets got a late score after an onside kick was returned inside the 20 and the Patriots then scored a garbage TD with seconds left).
2011: 21-17 to NY Giants (offense was bad for most of the game, except the end of the first half and beginning of the second, while the D played more than good enough to win)
2012: 28-13 to Baltimore (offense was shut out at home in the second half, while the defense eventually wore down allowing the Ravens to pull away and win convincingly)
2013: 26-16 to Denver (offense again was bad most of the day, until scoring two late meaningless TDs; defense gave up a ton of yards, but held the Broncos to enough field goals to where it could have been a game in the 4th had the offense played well)
2015: 20-18 to Denver (defense was just fine, allowing only 20 points, 7 of which came after an INT inside their own red zone; offense struggled mightily against the Broncos ferocious D).