I never watch this show but I am familiar with the humor he and Stewart employ. However, when I happened upon this clip, I couldn't believe how genuinely rude Colbert treats this author (David Levy, author of Love and Sex with Robots). Maybe some of his guests are able to just laugh it off. But this guy just looks offended. That doesn't stop Colbert though. He just keeps insulting the guy and being a grade-A douchebag.
There's a difference between poking fun and just being downright rude. And obviously Colbert has lost sense of the line that differentiates the two.