You can swipe to delete messages in the iPhone's Mail app, but you'll need to enable this option in Settings. By default, you'll only have the option to "Archive ...