Thanks for the advice. I don't care too much about the caching, so I just deleted the plugin.
After deleting the plugin, I saved permalinks twice, like you suggested. Since I no longer have a cache plugin, there is nothing to reset.
Pages still show "404 Page Not Found".
