The game opens with the young girl dragging an oversized attaché case toward a warehouse with difficulty. "Bloody" Harry Macdowell has just carried out a coup against Big Daddy, the leader of the Millennion organization, and his daughter Mika needs to find someone that can protect her and stop Harry's mad plans. The occupants of this warehouse include a kindly looking old doctor, and a man with a notable scar on his face. Mika arrives, and the man with the scar claims the contents of the case: two massive handguns. That man is revealed to be the game's title character Grave, and now that he is armed he can start his mission.
Gungrave first approaches its stages as a series of missions issued by Dr. T, first to gather information on the current makeup of Millennion from a low-level street gang, and next destroying a research facility that creates Harry's undead soldiers. In the third stage, while attempting to pump information from an informant, Grave comes into contact with the leadership of the Millenion organization—once friends and allies that he now faces as enemies. Each have used the research Harry supported to give themselves inhuman powers. From here on, Grave is hounded by each member as he makes his way to Harry's tower at the heart of the city.
As the player progresses, the game uses anime cutscenes to reflect on the history of young Brandon and Harry, gradually bringing the pair's back story into focus. Close friends, the two had both become lieutenants in the Millennion organization, working directly under Big Daddy, the group's leader. Brandon shared a bond with Big Daddy and some flashbacks show the two sharing more of a father-son relationship. Brandon even let Big Daddy marry the woman he loved so that she could find a better life, but the two remained close. Not content with the power he had been given, Harry asked Brandon to help him kill Big Daddy so that he could take over. When Brandon refused, Harry shot his friend in the left eye, killing him. Fifteen years later, Harry carried out his coup. His actions as leader inspired Dr. T to revive Brandon who was the only person capable of stopping Harry. Dr. T's connection to all of this is not made clear, but he often makes comments that indicate some connection to Brandon's former life.
Grave picks apart the leadership of Millennion to make his way to Harry. At the top of the tower that Harry uses as a headquarters, it is revealed that Big Daddy still lives in the form of a twisted monster. Harry forces Grave to fight his creation, and following the final battle, Harry accepts his defeat graciously and allows his friend to kill him.
With Harry defeated, Mika's protection becomes Grave's only concern, and to keep his promise to Big Daddy of protecting the family, Grave protects Mika while they drive away from this tragedy.