Horizontally just because I'm chicken.
LOL! If it says "marker", it automatically goes horizontal at my house. LOL! But I do know that they say it is OK to store them vertically.
What can I say! Old habits die hard and besides, this way they can go in a box and then in a drawer-AWAY FROM THE CATS!!!!!!! LOL!