A.G., you have entered a challenging situation given the inconsistencies your students have experienced and the fact that you are there mid-year. Being at a loss on your 3rd day is not surprising as it often takes teachers a while to get a handle on classrooms when they are there from day one. Give yourself permission to learn as you go, and stick with your resolve not to acquiesce to student demands that prevent them from learning.
Since the school is well-organized, I am guessing there are clearly established procedures regarding office referrals and the consequences of receiving one. If you don't have a written copy of them, be sure to get one so you can inform students of the requirements of your job and that you will be enforcing them. Leaving the classroom without permission is truancy and you should never ignore it but should follow the school procedures for dealing with it. This is a safety/liability issue; those students are to be under your supervision and when they are not, you are unable to ensure their safety.
I recommend that you learn every student's name as quickly as you can. The first year I taught middle schoolers, I did not make this a priority. The second year and every year after that, I took small-group shots of students, asked reliable students or office personnel to help me label them, and then I STUDIED hard every night until I knew students' names. Because this was a challenging task for me, it usually took me about 2 weeks to be completely comfortable with all those names. I took class time to test myself, and students appreciated the effort. It made an enormous difference in the classroom behaviors. I also purposefully learned the names of some well-behaved kids first and sometimes "forgot" names of students with behavior issues at the beginning so students didn't receive reinforcement for drawing negative attention to themselves, as in, "All the teachers know her name. She's always in trouble." Instead, with those students, I tried to notice something positive to attach to their name, such as,"Oh yes, I remember your name. You shared a pencil when Marvin didn't have one yesterday."
I also encourage you to reframe the classroom expectations with positive rather than negative language. For example, instead of "no cellphones", you might remind students that "Cellphones are to be turned off and put out of sight." Instead of "no walking around" you could tell a student "Right now you need to be in your seat." (It helps if you can also tell the student when he will be out of his seat, for example, "In ten minutes, students who have worked at least 4 problems will be allowed to compare answers with a partner.")
You would likely benefit from reading other questions and answers on this discussion board, even those that don't appear to be related to your issue or your grade level, because a response might trigger an idea that would work for your students.
Hang in there!