Stephen Dodds, who had preyed on the same victim previously, went into the 70-year-old's home last August and stole around £70 in coins he had saved in a tea caddie.
The 42-year-old, along with his accomplice Kim Mason, 52, then forced the pensioner into a car and drove them to his bank.
A neighbour who saw the kidnap said the victim looked "pale and faint" as he was led to the black Ford Fiesta.
Prosecutor Gavin Doig told Newcastle Crown Court the victim was driven to a cash machine by the couple and ordered to insert his card.
He added: "The victim put his pin number in and the balance he had was £200.
"Dodds pressed the buttons to withdraw £200.
"The victim told him he didn't want to, but Dodds pushed him to the side and made the withdrawal."
The victim was then taken back to his home in Blyth, where he lived with his partner.
When a police officer arrived at the house, the victim looked "nervous" and his "hands were shaking".
Mr Doig added: "The constable asked if he was OK, and he shook his head."
In an impact statement, the victim said: "Since the incident I am very dubious and frightened of leaving my home.
"I try to avoid leaving the house and feel I might bump into them again.”
Dodds was jailed for 17 weeks in 2015 for theft, fraud, harassment and assaulting the same victim.
This time, Dodds, of no fixed address, pleaded guilty to kidnap, robbery and theft.
He also admitted possessing a prohibited weapon and a blade, which were recovered from him in June 2019.
He pleaded guilty to possession with intent to supply heroin and possessing cocaine and diamorphine in relation to drugs found at his home.
Accomplice Mason pleaded guilty to kidnap and allowing premises to be used for the supply of class A drugs.
Dodds was jailed for six years and Mason for two-and-a-half years.
Ben Campbell, mitigating, said Dodds has made progress while in custody on remand and could stay out of trouble when drug-free.
Rachel Hedworth, representing Mason, said she was "utterly remorseful" for her role in the crime.