Well yeah, it’s football, rarely is anything solely one person’s fault. Brady gets a big chunk of the blame because given the situation, with just over a minute left before the half, the absolute last thing you can afford to do there is turn the ball over. And it wasn’t even just a physical mistake (underthrown ball) it was a mental one (Gronk triple covered).
Do you wish the defense had stepped up and stopped the Ravens? Of course, and yes, they got 20 yards on a somewhat ticky tacky pass interference call on Revis, who was in good position and looking back for the ball.
Still, the defense should never have been in that position to begin with so Brady has to shoulder a big chunk of the blame for that turn in the game.