Kurtis Dilks is accused of being part of a gang that used a sledgehammer to smash into Mr Cole and Sharon Canu's home in Surrey on 21 January 2020.
Mr Dilks told a jury he was "most certainly not" involved in the robbery.
At Nottingham Crown Court, nine other men are also on trial over a series of "ruthlessly executed" burglaries.
The majority of the charges relate to the theft of the £3.5m Portland tiara from the Welbeck Estate in Nottinghamshire.
Giving evidence on Monday, Mr Dilks said he told police he felt "sick" when he was informed his DNA was found on items from the scene and that he believed somebody was framing him.
Jurors were previously shown interview footage of Mr Cole telling police "I knew now, I am going to die", as he recalled masked raiders breaking into his home.
Mr Cole said his hands were tied while he was holding his young daughter and the gang threatened to cut his fingers off with pliers.
Mr Dilks' defence barrister, Simon Eckersley, asked the defendant if he went into Mr Cole's house or was if he was involved in the planning or commissioning of the robbery.
The 34-year-old answered "no" or "most definitely not" to the questions.
Mr Eckersley said the court had heard his DNA was found on a knife recovered outside Mr Cole's home on a walkway, and also on a cable tie that was found on the kitchen table.
Mr Dilks said: "The only aspect I can believe is those items were stolen out of my van.
"It's something I don't do every day, to check everything is there. I would only check if I needed to use the item."
Mr Eckersley also questioned him over his alleged involvement in the theft of the tiara, which he denied.
Mr Dilks, of Clifton, Nottingham, faces three charges of conspiracy to commit burglary, four charges of converting criminal property, three charges of conspiracy to commit robbery and two counts of robbery.
He denies all charges.
The trial continues.
* Matthew Johnson, 35, of Kingsthorpe Close in Nottingham. He is accused of one count of conspiracy to commit burglary and one count of converting criminal property
* Darren Stokes, 32, of Staunton Drive in Nottingham. He is accused of one count of converting criminal property
* Ashley Cumberpatch, 36, previously of First Avenue in Carlton, Nottinghamshire. He is accused of three counts of conspiracy to commit burglary, five counts of converting criminal property, and three counts of conspiracy to commit robbery
* Adrian Eddishaw, 34, of Northall Avenue in Bulwell, Nottingham. He is accused of one count of conspiracy to commit burglary and one count of converting criminal property
* Andrew MacDonald, 42, of no fixed address. He is accused of three counts of conspiracy to commit burglary, five counts of converting criminal property, and three counts of conspiracy to commit robbery
* Kurtis Dilks, 34, of Whitegate Vale in Clifton, Nottingham. He is accused of three counts of conspiracy to commit burglary, four counts of converting criminal property, three counts of conspiracy to commit robbery, and two counts of robbery
* Christopher Yorke, 50, of Rose Ash Lane in Nottingham. He is accused of one count of converting criminal property
* Gordon Thornhill, 49, of Mosswood Crescent in Nottingham. He is accused of one count of converting criminal property
* Tevfik Guccuk, 41, of Houndsden Road, Southgate, London. He is accused of five counts of converting criminal property
* Sercan Evsin, 27, of Meadow Close in Barnet. He is accused of four counts of converting criminal property