It seems my webhosting providers impose a 2MB limit on file uploads through PHP. It seems fair to me, but on the other hand if your running a website that requires your users to upload images, not all of them are going to understand the limits or understand how to reduce the size of the image to bring it under the 2MB limit.
Unless your webhosting provider gives you access to your own PHP configuration file, one solution is to override the PHP settings by creating an .htaccess file, which specifies the max filesize, and post max size.
If your using Linux create a new file on your desktop named .htaccess then copy paste the following code.
Save the file and upload this to your webspace. After this you should be able to upload files, images and other media larger than 2MB