Let's be honest. Presidential advisors testify all the time. They don't have the same responsibilities vis a vis Congress as members of the executive departments. But they can and do testify. There's only one reason why you agree to 'talk' to Congress unsworn, in private and without a transcript: because you want to be able to lie or dodge questions in a way that's too embarrassing to do in public.