I've tried on that sweatshirt and would take an M for it to be fitted. I'm really short-torsoed so it comes down to the top of my hips.
I have the fleece pants and they shrink in the wash, but I found that they stretched out a bit after wearing.