It did not take long for a night of drama to get going as Ollie O'Neill gave Orient the lead after just 145 seconds when he arrowed in a first time volley at the near post following Galbraith's clever lofted free-kick.
The visitors enjoyed a fairly dominant opening spell until Stockport started to grow back into the contest and struck the crossbar after 17 minutes through Fraser Horsfall's header from eight yards.
From then on, it was Stockport who took control and Horsfall - who scored with a headed effort in the first leg and caused trouble all evening - again went close, glancing narrowly wide of the far post.
League One's top goalscorer Charlie Kelman almost doubled Orient's advantage slightly against the run of play when he headed against the underside of the crossbar from close range shortly before half-time.
Stockport came out with real intent after the break and found a deserved equaliser when Tanto Olaofe sent a phenomenal back-heeled flick into the corner following Diamond's low effort with 16 minutes of the 90 remaining to force extra time.
County continued to create chances as Horsfall forced Keeley into two more fine saves.
Odin Bailey almost won it for the hosts with practically the last kick of the game but his placed finish when through on goal cannoned away off the post.
It came down to spot-kicks and Orient managed to hold their nerve, scoring all four of their penalties to go through.