1.15.2012

Block unwanted advertisements with /etc/hosts file on Linux

I surf the web an awful lot, probably slightly more than your average 13 year old geek. I notice that a lot of sites load rather slowly mostly because your waiting on content from outside the specific domain. For example if you go to a website like thechive.com (one of my favorites) you will notice it takes quite a long time loading the ads.

Although I mentioned thechive.com I spend most of my time on the net looking for information, not entertainment. These ads really hinder my search speed!

So here is a quick way you can block all the ads. Not only will your surfing be faster but you will also save some bandwidth.

First off I would like to thank the fine folks at http://winhelp2002.mvps.org/ for doing all the leg work and collecting all the data necessary for this to work.

Now this is simple.
First, let's make a copy of your current hosts file. Some people still use this!

NOTE: Everything done below should be done as root, or with sudo access.

Here we will make a copy of your /etc/hosts file and save it in your home directory in a hidden file called .etchosts , but you can save it anywhere you like. Just be sure to change the script to make the location of your file.
cp /etc/hosts ~/.etchosts Now we will make the shell script.
run:
vi /root/update_hosts.sh
Fill the file with the following:
#!/bin/bash
cd /tmp
wget http://winhelp2002.mvps.org/hosts.txt
rm /etc/hosts
mv hosts.txt /etc/hosts
cat ~/.etchosts >> /etc/hosts
Now we have to make sure the script is executable:
chmod +x update_hosts.sh So just a little explanation of what the script does, it's simple.
1) cd /tmp (Changes to a temporary working directory)
2) wget http://winhelp2002.mvps.org/hosts.txt (Gets the hosts.txt file from mvps.org and saves it.
3) rm /etc/hosts (Deletes the current /etc/hosts file)
4) mv hosts.txt /etc/hosts (Moves the new downloaded file to the /etc/hosts file)
5) cat ~/.etchosts >> /etc/hosts (Moves your old host entries back into the new file)
Now all you have to do is run the script and your advertisements will disappear from your browser.

You can use a cron job to update the hosts file automatically every night. The good folks at mvps.org update the file fairly regularly, not everyday, but a couple times a month.

Let's add it to roots cron.
run:
crontab -e
Then enter the following in the file.
59 23 * * * /root/update_hosts.sh
Remember, if you need to add something to your hosts file you now want to add it to ~/.etchosts and then run the script to update /etc/hosts.


REFERENCE: http://winhelp2002.mvps.org/hosts.htm

16 comments:

  1. Right up to running crontab, none of that should be done as root. You can do the earlier bit as a normal user. I'd recommend nano or jed rather than vi, much easier to learn.

    ReplyDelete
    Replies
    1. but basically, you don't have anything useful to add to the mans script, do you?

      Delete
    2. Encouraging users to eschew vi for lame editors like nano or jed (jed?) is a critical mistake. Failing to learn ex/vi may someday leave you stranded with an unsolvable problem if your system becomes hosed and editing a file is the only solution, but that system only has ex/vi installed. vi is the preeminent editor on ALL unix systems. But then, you really had nothing USEFUL to add to the mans script, did you?

      Delete
    3. I don't think I've ever seen one that has not nano as default. Whenever that's the case/there's no nano, then, well, one who does not know "vi" may have to use some other PC and learn the basics/make some cheatsheet in a piece of paper and then go back to the troubled machine.

      It's perhaps not the ideal scenario, but it's not like encouraging people to walk on African jungles without knowing how to shoot a rifle. Most of the time you can go by perfectly fine without ever learning vi, somewhat like you don't need to be a mechanic to have a car, and even to know how to do some basic maintenance.

      Delete
    4. You can do just about anything as root, as long as you're not doing anything stupid. I'd argue that's perhaps more risky to have a "sudo" than do everything (specially withoug password) than to use just "su". But I don't know for sure, I may be missing something. To me it seems that Ubuntu-style omnipotent sudo is more about convenience than safety. With some distros' original sudo config you wont even be able to do barely anything with just sudo by default, su is required, at very least to define what each user can sudo.

      Delete
  2. sudo su
    cat /etc/hosts >/etc/hosts.orig

    cat >/usr/local/bin/update_hosts.sh </etc/hosts
    cat /etc/hosts.orig >>/etc/hosts
    EOF

    chmod +x /usr/local/bin/update_hosts.sh

    crontab -e
    59 23 * * * /usr/local/bin/update_hosts.sh

    exit

    ReplyDelete
  3. Just a note of thanks for creating such clear, well written, helpful information. I've successfully implemented your instructions on my Centos 6.2 distro. Excellent results!

    ReplyDelete
  4. Sorry, how does this vi thing work.
    Having trouble here

    ReplyDelete
  5. vi is a command line text editor. You can also use gedit or any text editor you like. If you want to learn vi, read this:

    http://acms.ucsd.edu/info/vi_tutorial.html

    ReplyDelete
  6. KWrite is a popular text Editor, on KDE, and many Systems.

    ReplyDelete
  7. You could also use ABP on firefox or the same on Chromium.

    Regards.

    ReplyDelete
  8. You shouldn't remove your own/default /etc/hosts

    It should suffice to:

    wget http://winhelp2002.mvps.org/hosts.txt
    cat hotst.txt >> /etc/hosts
    rm hosts.txt

    ">>" will only add to the extant file

    Then you edit your hosts file and remove the redundant lines, such as

    127.0.0.1 localhost

    I don't know if it matters so much that there are those redundancies though.

    By not deleting your own hosts you won't lose some lines that may be setting the domain and whatnot. You might have lines somewhat like:

    127.0.1.1 debian-joejoejoe.WORKGROUP debian-joejoejoe

    Whatever it does, I'm not an expert on that.

    Those would have been erased in the other method, and things like samba or network folders might not work I guess.

    But if you're going to do that anyway, you only need to

    rm /etc/hosts/
    wget -O /etc/hosts

    no need to move and rename everything all around the place.

    If you want to make a script that does that periodically, you may want to consider to have a "hosts.header" (you name it) file with your current/default hosts file, and then have a script (called by cron) that will do something along the lines of

    wget -O /tmp/newhosts (or /dev/shm, wherever works)
    cat /etc/hosts.header > /etc/hosts (note the single, not double, ">", it erases the old file, so no "rm" is needed)
    tail -n +26 /tmp/newhosts >> /etc/hosts (now it's a double, ">>", as it's only adding new stuff to the end of the file)


    "tail -n +26" will trim cut out the 26 first lines of the downloaded hosts file, wich will be added to your own custom "header" via ">>".



    Always backup the relevant files before testing new things.

    ReplyDelete
  9. After following the manual, I see a ^M after every entry in /etc/hosts, like,
    127.0.0.1 ads.doubleclick.com^M
    Just wanted to know if its normal. Its working perfectly though.

    ReplyDelete
  10. @Anonymous - usually this occurs when the file was created on Windows. You can remove the ^M by using dos2unix.

    http://linux.about.com/od/commands/l/blcmdl1_dos2uni.htm

    ReplyDelete
  11. Thank you for the instructions. Worked well on my Linuxmint 16.

    ReplyDelete
  12. This is my solution, based on what I've seen in this thread. These instruction work in Ubuntu from a command prompt.

    1) Create a copy of your original hosts file to use as a header:

    sudo cp /etc/hosts /etc/hosts.header

    2) Create the script to update the hosts file:

    nano update_hosts.sh

    3) Enter the following text in nano:

    #!/bin/bash

    #Change to a temprary working directory
    cd /tmp

    #Download the current hosts.txt file
    wget -q http://winhelp2002.mvps.org/hosts.txt

    #Overwrite the hosts file with the hosts header
    cat /etc/hosts.header > /etc/hosts

    #Append the hosts.txt data to the hosts file
    tail -n +26 /tmp/hosts.txt >> /etc/hosts

    #Delete hosts.txt
    rm /tmp/hosts.txt

    #Exit the script
    exit

    4) Save the file with CTRL-X in nano

    5) Change ownership of the file to root and make the file executable:

    sudo chown root:root /path/to/update_hosts.sh
    sudo chmod 700 /path/to/update_hosts.sh

    6) Ad an exception for update_hosts.sh to sudoers

    sudo visudo

    7 Below the line that reads '%sudo ALL=(ALL:ALL) ALL', enter the following:

    user ALL=(ALL) NOPASSWD: /path/to/update_hosts.sh

    *Replace 'user' with your username

    8) You should now be able to execute the script without a sudo password by entering the following:

    sudo ./path/to/update_hosts.sh

    ReplyDelete