DM 171: Automated Site Map with

Site Maps

Site maps are a great way for your users to find what they need if they have any problems with your navigation scheme; however, remembering to update the site map every time you add or remove a page can be annoying. How about a solution that solves your problem automatically!

A great free CGI script called is available here:

You will need to download the compressed file via the link at the bottom of the page. Once downloaded, uncompress the file. You will find a folder called “tree” with several files in it; read the README (of course), is the CGI script written in the Perl language, tree-template.html is a sample file to display your results, and COPYING is a file that explains the GNU GENERAL PUBLIC LICENSE. You will be installing the CGI script for your site map.


Follow these instructions exactly…

  1. Open with a plain text editor like Wordpad or TextWrangler. You will need to make sure that your editor will work with Unix carriage returns (line endings).

  2. Note that the first line reads: #!/usr/bin/perl -w
    Every CGI script written in Perl has to know where the Perl interpreter is located within the server, this is known as the “path to perl.” #!/usr/bin/perl is the standard path for most servers and does not need to be changed for The -w turns on “warnings” for programmers and does not need to be changed. Your mileage may vary on other servers, consult your documentation and/or administrator!

  3. On line #29 of, set the file to run as a CGI script, it will look like this: $conf::cgi = 1;

  4. On line #30 of, add the server path to your web directory for the $conf::basedir_cgi variable value:
    $conf::basedir_cgi = "/home/s65428/your-webhawks-user-name/public_html";
    The line for my example looks like this:
    $conf::basedir_cgi = "/home/s65428/dm171wally/public_html";
    This lets the script know which directory to make a site map for.

  5. On line #31 of, make $conf::templatefile = "tree-template.html"; look like:
    $conf::templatefile = "";
    This lets know that you will not be using a template file.

  6. On line #32 of, make it look like this: $conf::baseurl = "/~your-webhawks-user-name/"
    For my example, it looks like:
    $conf::baseurl = "/~dm171wally/";

  7. Make line #39 look like this:
    @conf::defaultfiles = ('index.html', 'index.htm', 'index.php', 'index.shtml');
    You will be adding index.shtml to the values so will look for that file as a default file too.

  8. Optional! Add a pound sign (#) to the beginning of line #44 to comment the line and remove the pound sign from the beginning of line #46 to uncomment the line. This will limit to only listing regular web files. You may also add any other file extensions you wish.

  9. Lines #56 through #60 give instructions for directories and/or files to be included in the site map, the default is set to everything. Adjust to your preferences.

  10. Lines #62 through #67 are used to configure which files and/or folders to be excluded from the site map. Adjust to your preferences. I will usually add in a value like '*hide*' which allows me to add the word "hide" to any file in order to exclude it from the site map. Very handy for development; the asterisks act as "wild cards" so that "hide" anywhere in the file or folder name will exclude it from the site map.

  11. Lines #69-#73 configure your site map links, configure to taste.

  12. Lines #101-#120 configure the output of if you are not using a template file. Note that the default is set up to write an entire html file, skeleton and all, so you may want to configure this differently. Whew, this should be the last of your changes to

  13. Create a folder named "cgi-bin" inside of your webhawks public_html folder but not inside any other folders. This folder will hold and CGI scripts that you will have running on your site. Other servers than may have a cgi-bin already established, consult your admin.

  14. Upload your modified copy of to your cgi-bin and set its permissions to "755" to tell the server that this script is OK to run. Make sure that you upload it as “text,” you will probably need to use an FTP client like Fetch to do this.

  15. Create an .shtml file for your site map (I call mine sitemap.shtml).

  16. Use a server side include to include the file in your site map page, mine looks like this:
    <!--#include virtual="cgi-bin/" -->
    The important thing is that the path is correct.

  17. Upload your site map HTML file to the webhawks server and view it with your browser.

  18. If you are lucky, you will see a nested list of links to your files. If not, it is time to troubleshoot…
    1. Was edited with a text editor that will preserve its Unix line endings?
    2. Was the folder “cgi-bin” named correctly and uploaded to your public_html root?
    3. Was uploaded as a text file? You can force most FTP clients to do that if needed.
    4. Were the permissions changed to 755 after was uploaded to the server?
    5. Does your site map file have a .shtml suffix so your include will work?
    6. Is your include instruction well formed and the path to correct?

  19. If it still doesn't work, you may need to start over from scratch (I did) just in case the file has been corrupted in some way.

Installing a CGI script can be frustrating, there is a lot that can go wrong. However, when it works, you rarely need to touch it again.