That is one of the awesome qualities of nitrous though...so one hit doesn't last long at all. This means you can always be positive that in about three minutes you'll feel just as you did before you inhaled it. If you want to prolong it, just inhale more and stagger them.
It is just another one of those things that people hold their misconceptions about; apprehensive it could cause them brain damage. Consider that nitrous has been used since 1844 is still used widely in surgeries and during dental procedures. This is because it is safe and effective. Obviously using it for recreational purposes isn't as 'safe' as if a dentist or surgeon was administering it. But taking an hour or so out of your busy, stressful day to inhale some whippets wont incur you any damage and it is unlikely to get you in trouble with the law or the probation officer.
As far as I am concerned people can inhale whatever the hell they think is going to make them feel better about the ridiculous world they live in.