The best way to spider & download an entire website

There seem to be a lot of GUI tools out there that purportedly download entire websites. The last one of these I tried using was SiteSucker, which sucked pretty badly. What you really need is this:

wget --recursive --no-clobber --page-requisites --html-extension --convert-links --domains mywebsite.com --no-parent www.mywebsite.com

run from the terminal. I put this here in the interests of me remembering all the options and hopefully someone else saving themselves so many boobless hours of crap applications that don’t work.

Continue reading “The best way to spider & download an entire website”