Brown said he was worried about what would happen to the nation if too many people used the drug too often.
"And all of a sudden, if there's advertising and legitimacy, how many people can get stoned and still have a great state or a great nation?" he asked. "The world's pretty dangerous, very competitive. I think we need to stay alert, if not 24 hours a day, more than some of the potheads might be able to put together."
Watch part of Brown's interview via NBC: