It was a bit hidden in Chrome, but found it:
Settings -> Advanced (bottom of the page) -> Privacy: Content settings button -> Cookies: Manage exceptions button

Then add [*.]youtube.com and set it to block.