Kicking my roommate out of unit

I have been living with my roommate for over a year now. But, I started to notice that my roommate is using my shit and even taking my money from my room. I noticed it about 6 months ago but now it starts to rack up and piss me off a lot and I want to get rid of him. Of course, I don't have any physical evidence that he is taking my stuff but I do know that most of the items that I use (such as shampoo, food, toilet paper, etc. and even money!) that I know I did not use as much and are gone by the time I am trying to use it.

To explain the lease situation, We live in a one-bedroom apartment where the lease is under my name and my roommate did not sign anything with the landlord. My landlord just knows that he is living with me and he pays me the rent and I deal with the landlord along with paying hydro bills and internet bills. When he was coming in, he just paid me the first and last month of his portion of the rent and we did not sign anything because he was introduced me as my mom's friend's friend so I thought I could somewhat trust him but I guess not.

I read this somewhere that as long as I give him 'reasonable' notice before the actual eviction, I can kick him out without any legal issues as he did not sign anything legally. I wonder if anyone has knowledge of this situation.

Thanks in advance.