Wget Command: What It Is and How to Use It (12 Examples Included)

This guide teaches you how to use the wget command in Linux. It provides 12 examples of wget commands in action to help you use them. Once you finish, you will know everything about wget and how to use it to fetch files from the web.

What is the Wget Command?

Wget is a computer tool created by the GNU Project. You can use it to retrieve content and files from various web servers. The name is a combination of World Wide Web and the word get. It supports downloads via FTP, SFTP, HTTP, and HTTPS.

Wget is written in portable C and can be used on any Unix system. It can also be run on Mac OS X, Microsoft Windows, AmigaOS, and other popular platforms.

How to Install Wget

For this demonstration of the wget command, we will use Ubuntu 16.04. But the syntax will work on any other Linux distribution.

First, access your server via SSH:

ssh user@your_server_ip -p port

To install wget on Ubuntu 18.04 or similar, run the following command:

sudo apt-get install wget

To install wget on CentOS 7 or earlier distributions, use:

sudo yum install wget

Once the installation is complete, you will be ready to use it. Also, knowing basic SSH commands can make things easier.

Wget Command Examples

To start, here are 12 examples of wget commands you can use for daily tasks. Keep in mind you can also call these from scripts and cron jobs.

1. Using Wget to Download Individual Files

One of the basic wget command examples is downloading a single file and saving it to your current working directory. For example, you can get the latest version of WordPress with the following command:

wget https://wordpress.org/latest.zip

In this example, a file named latest.zip will be downloaded to the current working directory. You will also see additional information such as download progress, speed, size, date, and time.

2. Using Wget to Download Multiple Files

We can take wget usage a step further and download multiple files at once. To do this, we need to create a text file and place the download URLs in it.

For example, we will retrieve the latest versions of WordPress, Joomla, and Drupal with wget. Enter the following:

nano example.txt

This will create a file named example.txt and open a text editor interface. Paste these links inside:

https://wordpress.org/latest.zip
https://downloads.joomla.org/cms/joomla3/3-8-5/Joomla_3-8-5-Stable-Full_Package.zip
https://ftp.drupal.org/files/projects/drupal-8.4.5.zip

Once done, you can use -i to get all files stored in your example text file:

wget -i example.txt

Wait for the process to finish, and you will have the installations of three of the most popular content management systems.

3. Using Wget to Save Files with Different Names

In this wget example, we will save a file with a different name using the -O option:

wget -O wordpress-install.zip https://wordpress.org/latest.zip

In this case, the downloaded resource will be saved as wordpress-install.zip instead of its original name.

4. Using Wget to Save Files in a Specified Directory

You can use wget to place a file in another directory using the -P function:

wget -P documents/files/ https://wordpress.org/latest.zip

The file retrieved with this syntax will appear in the documents/files/ folder.

5. Using Wget to Limit Download Speed

With wget, you can also limit the download speed. This is useful when fetching large files and prevents it from using all your bandwidth. This example sets the limit to 500k:

wget --limit-rate=500k https://wordpress.org/latest.zip

6. Using Wget to Set Retry Attempts

Internet connection problems can interrupt your download. To address this, we can increase retry attempts using the --tries function:

wget --tries=100 https://wordpress.org/latest.zip

7. Using Wget to Download in the Background

For extremely large files, you can use the -b function. It will download your content in the background:

wget -b http://example.com/large-file.tar.gz

A wget-log file will appear in your working directory, which can be used to check progress and status. You can also use the tail command:

tail -f wget-log

8. Using Wget to Download via FTP

The command can also be used with FTP. You only need to specify the username and password, as in this example:

wget --ftp-user=YOUR_USERNAME --ftp-password=YOUR_PASSWORD ftp://example.com/somefile.tar

9. Using Wget to Resume Interrupted Downloads

Your download can be interrupted if you lose internet connection or experience a power outage. This is common with large files. Instead of starting over, you can resume the download using -c:

wget -c https://example.com/very-large-file.zip

If you proceed without -c, the new file will have .1 added at the end because the original already exists.

10. Using Wget to Retrieve Complete Websites

It is also possible to use wget to download the content of an entire website. This allows you to view it locally without an internet connection. Example:

wget --mirror --convert-links --page-requisites --no-parent -P documents/websites/ https://some-website.com

Command breakdown:

• --mirror: Makes the download recursive.

• --convert-links: Converts all links for proper offline use.

• --page-requisites: Includes all required files like CSS, JS, and images.

• --no-parent: Ensures directories above the hierarchy are not retrieved.

• -P documents/websites/: Ensures all content goes to the specified directory.

Once finished, you can open the downloaded website locally

and find all files in the documents/websites/ folder.

11. Using Wget to Find Broken Links

We can use wget to locate all broken URLs showing a 404 error on a specific website. Run:

wget -o wget-log -r -l 5 --spider http://example.com

• -o: Saves output to a file for later use.

• -l: Specifies recursion level.

• -r: Makes the download recursive.

• --spider: Sets wget to spider mode.

Now you can check the wget-log file to find the list of broken links:

grep -B 2 '404' wget-log | grep "http" | cut -d " " -f 4 | sort -u

12. Using Wget to Download Numbered Files

If you have numbered files or images in a specific list, you can easily download them all with:

wget http://example.com/images/{1..50}.jpg

Conclusion

We hope this guide has been helpful! 🚀 Remember, don’t hesitate to ask questions on the HolyHosting Discord or contact our support team.

Follow us on Twitter @HolyHosting to stay up to date.

Stuffy @ HolyHosting