64 bit - 32-bit operating system on 64-bit machine with 4 gigabytes of memory or less

I'm going to be buying a laptop soon. I'm going to install Linux so that it will dual-boot Windows and Linux. If it is a 64 bit machine with 4 gigabytes of memory or less, would there be any downsides in installing a 32-bit copy of Linux instead of a 64-bit one?

If it's relevant, I will most likely be installing Xubuntu 12.04.