Mysql running out of memory

I am having a lot of trouble with mysql eating too much ram on a small machine and am looking for some help in figuring out why it is eating so much.

I have a small virtual machine with 256MB of RAM running Debian Wheezy. On this server I have apache2 and mysql installed. I don't do very much on this server, only a few lightly used websites and a mail server.

For some reason, several times a day, my mysql server crashes. When I check my syslog I find the following error:

kernel: [3323593.630722] Out of memory in UB: OOM killed process 9471
(mysqld) score 0 vm:327824kB, rss:37080kB, swap:0kB

So as far as I can tell, mysql starts to eat up too much memory and is killed by the system. I log slow queries and keep a tab on my mysql.err log but I dont see anything of much value in those that would show me why mysql starts to eat so much memory.

My my.cnf file has these options set:

key_buffer              = 8M
max_allowed_packet      = 16M
thread_stack            = 128K
thread_cache_size       = 8

query_cache_limit       = 512K
query_cache_size        = 8M

The other thing is that when I check the amount of memory being used when I start mysql and even during the day while it is running, I usually have about 128MB free. I don't see how mysql would end up eating that amount ever.

What can I do to track this problem down?