I use the baby oil with an old SU blender pen, but I do usually lie all my colors down, them basically color with the oil to blend it all together. This works with the SU as well as the Prisma pencils. I have found that doing it this way, I don't have as much trouble with uneven shading.
If you are using SU Whisper White, you really have to use a light hand because it is not a heavy cardstock. My technique also works better on this, because you aren't layering so much.
As with most things, start with less and you can always add more, if it is color or oil.
I think we all know it is to be our own worst critique! WALK AWAY! LOL