BannerFilter

Tired of all the flashy ad banners on the Web? If you run Squid, then BannerFilter is for you! And yes, it's Free.

Be sure to run the update.sh script to download the latest data files, since I update those a lot more often than I update the tarball.


Check banners on the Web!

There is a Web interface to BannerFilter, so you can check to see whether a particular URL would be blocked, without installing BannerFilter. If you have a banner that isn't blocked and you think it should be, check it here before e-mailing it to me!


ReadMe

BannerFilter v1.31
Copyright © 2004 Andy Lyttle
E-mail: bannerfilter@phroggy.com
Web: http://phroggy.com/bannerfilter/

Purpose:

This script blocks advertising banners on the Web. If you have a slow Internet connection, it will speed up your surfing, since the banners are not downloaded from the Internet at all.

Features:

Unlike most other similar solutions, BannerFilter does more than simply block advertising graphics. It also blocks advertising frames on a few sites, and automatically closes pop-up windows. A script can automatically download updated data files off the Web, and notify you when a new version is available.

Requirements:

A working installation of the Squid proxy server, available from http://www.squid-cache.org/, Perl, and a web server. If you don't have Squid already installed and running, do that first! BannerFilter should work on any platform that can run Squid and Perl; Windows users should see Windows.txt for additional information about that platform.

Quick Installation:

  1. Unzip the bannerfilter-1.31 directory in a useful place on the machine that runs Squid. You may want to put it under the squid directory, e.g. /usr/local/squid/bannerfilter-1.31.
  2. Move the contents of the www directory so they are accessible at http://127.0.0.1/bannerfilter/. If you want to put them somewhere else (which you'd need to do if your web server is not the same machine that runs Squid), edit bannerfilter.conf and give the correct URL.
  3. Test redirector.pl by running the script, typing something, and making sure you get something back (generally the same thing you typed). Press Ctrl-C to stop. If you get an error, fix it before proceeding.
  4. Add a line to squid.conf:
    redirect_program /path/to/redirector.pl
    For example:
    redirect_program /usr/local/squid/bannerfilter-1.31/redirector.pl
    Then restart Squid.
  5. Go to http://bannerfilter.internal/test.html from one of your client machines and see if it works. If not, figure out why.
  6. Run the update.sh script to make sure you have the latest version of the data files (see below). I update the data files far more often than I update the tarball, so what's included with the release you downloaded is probably pretty out of date. See below.

Configuration File:

The configuration file, bannerfilter.conf, may be kept in the same location as redirector.pl, or may be moved to wherever you'd like to put it. The redirect script will look for it in all the reasonable places I could think of; if you want to put it somewhere else, you can specify the path to the configuration file on the command line (in squid.conf) with the -c option, for example:
redirector.pl -c /mnt/config/bannerfilter.conf

The file contains configuration directives in the form "label = value"; blank lines and lines beginning with # or ; are ignored. The label is case-insensitive and whitespace before and after the = is optional. All the configuration directives should be explained by the comments in the file.

Updates:

To download the most recent updates to the data files, simply run the update.sh script from the same directory as the data files. If you'd like, you can automate this with a nightly or weekly cron job (be sure to cd to the appropriate directory first, then run ./update.sh). Sometimes new versions of the data files require that you upgrade to a new version of BannerFilter; if an update is required, update.sh will notify you. If the files have changed, the script will overwrite them and tell BannerFilter to reload the data files via “killall -HUP redirector.pl”. If you're running update.sh as a non-root user (other than the user Squid is running as), or if you don't have killall, you'll want to set RELOAD=0 in update.sh and take care of that yourself. Also, if you don't have lynx, set CMD to your preferred HTTP downloader.

Customizing:

If there are certain URLs that you would like blocked that aren't listed, you can edit the local.data files. These will not be overwritten by the update script. If there are URLs that you want never to be blocked, you can list them in exceptions.local.data.

Wildcards:

BannerFilter has (rather poor) support for wildcards. URL substrings in the data files can contain an asterisk (*), and anything that contains both the part to the left of the asterisk and the part to the right of the asterisk will be blocked. I will be improving this eventually.

Feedback:

If you find banners that should be blocked and aren't, legitimate pages that are being blocked and shouldn't be, or if you just have comments or questions, e-mail me at bannerfilter@phroggy.com. Please do not ask me for help installing Squid. Before you e-mail me about URLs you'd like to have added or removed, go to http://phroggy.com/pages/checkbanner.html and paste the full URL in there to make sure I haven't already taken care of it.

Thanks:

Thomas B. Fox and Miroslav Pragl suggested the local.data files. Dan Everton suggested the transparent 1x1 GIF option. Marcel J.E. Mol submitted a patch with minor code improvements. Aaron D. Marasco suggested several optimizations and improvements. Steven Lynn suggested making the DATA configuration directive optional, looking in the same directory as the script by default. Frequency suggested Getopt.


This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.


Windows

Notes on Windows support for BannerFilter 1.31

BannerFilter runs on Microsoft Windows with SquidNT and ActivePerl. This document is intended to suppliment, not replace, the standard README file. You can open the README file in WordPad, or any other text editor which can read UNIX line endings (i.e. not Notepad).

SquidNT may be obtained from:
http://www.acmeconsulting.it/pagine/opensource/squid/SquidNT.htm

ActivePerl may be obtained from:
http://www.activestate.com/Products/ActivePerl/

Installation:

Unzip BannerFilter and place it in the C:\squid directory, or anywhere else you'd like. StuffIt Expander (http://www.stuffit.com/) and recent versions of WinZip (http://www.winzip.com/) should both handle tar-gzip files.

Add this line to C:\squid\etc\squid.conf:
redirect_program C:/perl/bin/perl.exe C:/squid/bannerfilter-1.31/redirector.pl

Notice the forward slashes (/) instead of backslashes (\), and be sure to enter that all on one line. Substitute the correct locations of perl.exe and redirector.pl if you've put them somewhere else.

Configuration:

To edit bannerfilter.conf or the data files, you'll need to use a decent text editor that handles UNIX line endings, or open the files in WordPad which will convert them to Windows line endings (once you've saved them in WordPad you can re-open them in Notepad if you prefer).

For file paths in the configuration file, you can use either forward slashes (/) or backslashes (\).

If you're using IIS as your web server, you can either move the contents of the www directory to C:\Inetpub\wwwroot\bannerfilter, or create a virtual directory pointing to C:\squid\bannerfilter-1.31\www. Of course you can do something similar in Apache or whatever web server you're using.

Updates:

The update script is not officially supported on Windows, but update.sh can be made to work using Cygwin (http://www.cygwin.com/). You'll need to install wget, or something else that can do HTTP downloads, and turn off the RELOAD option in the update script (or change it to something that works). If SquidNT is installed as a service, you can restart it with:
net stop squidnt
net start squidnt

Acknowledgements:

Thanks to Andre Swanepoel for inspiration and ideas.


Mac OS X

[Mac]

If you don't already have Squid installed and configured, do that first. I recommend SquidMan.

Download the Mac OS X Installer package, which will install the various parts of BannerFilter to these locations:

Make sure the Personal Web Service is enabled. In System Preferences, click Sharing, and under the Services tab check the box for Personal Web Sharing. Then, add this line to your squid configuration (if you use SquidMan, go to Preferences and click the Template tab):

redirect_program /Library/BannerFilter/redirector.pl

Read the README file for more information. Of course, you can follow the standard UNIX installation procedure if you prefer, instead of using the Installer package.


Changes

Changes since 1.3:
  • Switched to using Getopt::Std to parse command-line arguments
  • Now supports -v --version -h --help options
  • Reading from STDIN is now done with <STDIN> instead of <>
Changes since 1.21:
  • Cleaned up update.sh and added several enhancements
  • update.sh now preserves permissions and ownership when overwriting data files
  • update.sh will only overwrite data files if they have actually changed
  • update.sh will only tell BannerFilter to reload the data files if they've changed
  • There's now an option in update.sh to not tell BannerFilter to reload
  • Improved error messages in redirector.pl; added helpful hints
  • Lines containing only whitespace in bannerfilter.conf will now be ignored
  • Semicolons now accepted as comment characters in bannerfilter.conf
  • Improved cross-platform support, CRs ignored in config & data files
  • Will now look for config file in more locations including same dir as script
  • Will now look for data files in same dir as script if not specified in config
Changes since 1.2:
  • Fixed the same reload bug I fixed in 1.1, but for exceptions.data this time
  • Added brackets around the -c option in the help line
Changes since 1.11:
  • Now uses a configuration file instead of setting variables in the script
  • Implemented half-assed wildcard support
  • Added exceptions.data files
  • Fixed a bug in logging; it wasn't writing the correct info to the file
  • Changed the log format; new format is “URL matches STRING”
  • README file updated to reflect these changes
  • Option for curl in update.sh now uses curl -s
  • update.sh can now handle multi-line version.txt files
  • Data files now include a modification date (yyyymmddhh)
Changes since 1.1:
  • Fixed a bug causing banners.data and popups.data to be ignored (oops)
Changes since 1.06:
  • Fixed a major bug causing banner lists to never be purged from memory
  • The local.data files are now required to exist, even if empty
  • Lots of optimizations and tweaks
  • Removed the extra file added in 1.06; it's really not needed at all
Changes since 1.05:
  • No significant code changes
  • Added one new file in the www directory
  • A ton of updates to the data files
Changes since 1.04:
  • Update script can now use either wget or lynx
  • No longer matching regex patterns, so no more weirdness (whoops)
  • Now using foreach loops instead of for loops; should be faster
  • A few changes in the www directory
Changes since 1.03:
  • Log file now puts each entry on one line, for easier grepping
  • Minor code improvements to redirector.pl
Changes since 1.02:
  • More banners (gee, big surprise)
  • Can now specify a 1x1 transparent GIF to use instead of the word "Blocked"
  • Added some new things to frames.data, and added new files in www
  • Added local.data files
Changes since 1.01:
  • Added lots more banners
  • Removed some URL substrings that sometimes have JavaScript code
  • redirector.pl doesn't loop through all the data files if it finds a match early
  • Data files are now read immediately, instead of after the first URL is entered
  • Added an empty flash.swf file for Flash-based banner ads
  • Added optional logging so you can keep track of what banners are blocked and why
  • Adjusted height of Yahoo frame page
  • update.sh now checks for updated versions of BannerFilter
  • Assorted documentation changes
Changes since 1.0:
  • Added more banners
  • Fixed the comments in the data files
  • Added update.sh to download new data files automatically
Changes since 0.9:
  • Moved the frames stuff out of the perl script and into their own file
  • redirector.pl now responds to HUP signals
  • Data files can now contain comments beginning with #
  • Added a special graphic just for goatse.cx