Ryan Adams and the Shining visited Late Show with David Letterman for the final time as the host readies to retire on May 20. Adams, who had some fun on Twitter reminiscing (below) about his fourteen years on the show, reprised a performance from 2002 as he dug into the Demolition cut “Starting to Hurt,” which had never been played with this band until now. The energetic performance was so good, in fact, that Letterman asked Adams to play the show out with an encore of the same song. 

Last night’s Letterman was special on the musical front for more than just Adams, as Letterman invited musical director Paul Shaffer into the guest chair to reminisce about some of their favorite memories over the years. A visibly emotional Shaffer shared stories about James Brown, Bob Dylan and even shared a special music video of “On Broadway” featuring appearances from Lorne Michaels, Martin Short and more.

Thank you @Letterman for all the laughs, the love and the memories. 14 years of it. I will miss you so very much.
XO pic.twitter.com/I3YGuFaU9H

— Ryan Adams (@TheRyanAdams) May 14, 2015

Replay the performance below. 

For comparison, here’s the tune from 2002.