SiteSearcher Documentation

Version 1.20

September 15, 1999

Copyright 1999 Zackware

Table Of Contents

• Disclaimer
• Purpose
• How It Works
• Limitations
• Un-Installation
• Glossary
• User Interface
• Site Pane
• File Pane
• Menus/Toolbar
• Options
• Contact Information
• Distribution

Disclaimer of Warranty

THIS SOFTWARE AND THE ACCOMPANYING FILES ARE SOLD "AS IS" AND WITHOUT WARRANTIES AS TO PERFORMANCE OR ANY OTHER WARRANTIES WHETHER EXPRESSED OR IMPLIED. NO WARRANTY OF FITNESS FOR A PARTICULAR PURPOSE IS OFFERED.

Good computer software procedure dictates that any program be thoroughly tested with non-critical data before relying on it. The user must assume the entire risk of using the program.

Purpose

Have you ever tried searching the web for a particular file, only to quit in frustration after wading through twenty web pages? Or, do you know of a web site that has a lot of great content, but you don’t want to take the time to download everything from that web site manually? SiteSearcher was designed to solve these problems. SiteSearcher will search web sites for the files you specify, automatically following links to other web sites. Files can be specified by extension, so it is possible to search for ZIP files, JPG files, or any other type of files. When SiteSearcher finds a file, it can be downloaded automatically, or you can download it manually.

SiteSearcher uses many features of Windows 95 (and above) to make this process as streamlined as possible. It is possible to be exploring 5 web sites and downloading 200 files at the same time, with no interruption to SiteSearcher, or any other running applications.

We feel that you’ll find SiteSearcher an indispensable tool for mining the web.

How It Works

SiteSearcher works by basically reading the code that makes up web pages. When you tell SiteSearcher to "browse" a web page, it downloads the code that makes up that web page, and looks for two things. It first tries to find any references to a file. If it finds one, and the file matches all the criteria that you have specified, that file is a candidate for download. The second thing that SiteSearcher does, is search for references to other web pages. When it finds one, it looks at a different set of criteria to determine if that web page should be added to the list of valid web pages.

On the most basic level, SiteSearcher will just add all the files and web pages it finds to a list that you can see in it’s window. You can then decide to either download the files, or browse the web pages that it finds. If you set up some of the more advanced features of SiteSearcher, you can have it automatically follow the web pages it finds, and also automatically download the files it finds. You have a substantial amount of control over these aspects. You can specify how many sites to automatically explore, and you can specify exactly what kind of files you’d like to download.

Limitations

• SiteSearcher will not try to understand any Javascript or Java code.
• SiteSearcher will only transfer files using HTTP. FTP connections will fail (FTP support will be added later).

Un-Installation

If you have installed SiteSearcher using the provided install program, there will be an item in the Zackware start menu item that allows you to un-install SiteSearcher. Only the executable and documentation files will be removed. Any files you have downloaded using SiteSearcher will remain on your hard drive.

Glossary

Here are a few words that this document will use that has special meaning for this application.

• Browse/Browsing: When SiteSearcher is searching a site for files or other web sites, we say that SiteSearcher is "browsing" the site.
• External Link: A link to a site that exists on a different server then the site that references it. For example, a link to www.yahoo.com from www.disney.com would be considered an external link.
• Local Link: A local link is a link that points to a site on the same server.
• Site: SiteSearcher considers a site the same as a web page, or a URL. Example sites: www.yahoo.com, www.disney.com.

User Interface

This is a screen shot of SiteSearcher in action.

This screen is broken into two main areas or "panes". The left pane will hold a list of all web sites that have been found, and the right pane will show all files that have been found that match your criteria. These panes are described in detail below. What items show up in these panes are dependent on how you have many of SiteSearchers options set up.

There is also a typical Windows menu bar, a toolbar, and a status bar. The menu bar and all it’s items will be discussed below. The toolbar has several buttons on it that act as shortcuts to many of the menu items. They will not be discussed in detail. Instead, refer to the menu item descriptions to learn what they do.

The status bar will display useful information about the progress of SiteSearcher. Near the right side of the status bar are three small boxes that also convey status information. The first area will have a progress bar showing SiteSearcher's activity. This allows you to see that SiteSearcher is in fact doing something. The next box shows how many web sites SiteSearcher is browsing. The last box shows how many files are being downloaded by SiteSearcher.

Next, we go into more detail about the two main portions of the interface.

Sites Pane

This area of the window show various information about the various sites that SiteSearcher has or is searching. Any site listing that has a + in front of it can be expanded by left-clicking on the +, or by double clicking on the line itself. When you tell SiteSearcher to explore a web site, it will add any other web sites that it finds underneath the original web site.

There are three possible icons that show up in front of the site name that represent the state of that site.

  This site has not been browsed.

  This site is currently being browsed.

  This site has already been browsed.

It is also possible to bring up a context sensitive menu by right clicking on the list of sites. This menu has some of the same entries that the "Site" menu has. Those items are described below in the "Menu Bar" section.

If you double click on a site, SiteSearcher will browse that site using the default mode.  Also, you can use the <INSERT> and <DELETE> keys on your keyboard to add and delete sites.

Files Pane

This area of the SiteSearcher window shows information about files that have been found that match your file search criteria. It may also show the size of each file that it has found, but has not yet downloaded. It is possible to select files from the file list area for downloading. The normal Windows extended selection operations work in this window. You can "shift click" to select several sequential files, or use CTRL keys to select multiple non-sequential files.

The file size column of this pane may or may not be filled, depending on how you have your options set up. In some cases, it is not possible to find the size of a file before SiteSearcher downloads it. In this case <Unknown> will show up in this column. Other times, SiteSearcher may not try to determine the file size, and in this case this column will be blank. Otherwise, this column will show the number of bytes the file is in length.

It is also possible to bring up a context sensitive menu by right clicking in this area of the window. This menu has some of the same entries that the "File" menu has. Those items are described below in the "Menu Bar" section.

If you double click on a file, SiteSearcher will add that file to the internal download list.  You can also use the <DELETE> key on your keyboard to delete any selected files.

Menu Bar

SiteSearcher’s menu bar contains several sub menus that allow you to access the many features of SiteSearcher. These menus will be discussed in the order that they show up in the Menu Bar. Note that many of these menu items can be accessed by right-clicking in various portions of the window, or by using the toolbar.

File Menu

This menu contains primarily items related with files that SiteSearcher has found. These items are described below:

• Download Selected: This item starts a sequential download of any selected files in the file list. These files are downloaded in reverse order that they are added to the download list. These files will be downloaded to a directory named "DL" under whatever directory you installed SiteSearcher to.
• Open...:  This item will run your default browser, and point it to the first file you have selected in the file pane.
• Remove Selected: This item removes any selected files from the file list. You can also press the "Delete" key on the keyboard to do this.
• Select All: This item selects all files in the file list.
• Determine Size:  This item will try to determine the file size of the files you have selected in the file pane.
• Find in List…: When selected, this item displays the following window:

  

  In the "Find" field, enter the text you wish to search for. SiteSearcher remembers several of your past entries, and you can select from them by pressing the small down arrow to the right of the text entry area.

• You can also specify if you only want to find one item in the file list, or all files that contain the text that you entered.

  When you are done making your selections, press "OK" to do the search, or "Cancel" to abort.

  If SiteSearcher finds one or more files, it will automatically select them. Finding files this way is useful when used in conjunction with the "Download Selected" or "Remove Selected" menu items.

• Cancel Download: This cancels any on-going file downloads, and re-add any unfinished downloads to your file list. Note that any partial downloads will be terminated immediately, possibly resulting in useless files.
• Exit: This item exits SiteSearcher.

Site Menu:

This menu contains items relating to searching web sites. These items are described below:

• Add…: This item (or pressing "Insert" on the keyboard) brings up the following window:

  

  In this window, you can specify a new site for SiteSearcher to explore. It is also possible to select from a list of previously entered sites by selecting the small arrow to the right of the text area.

  The three choices below the site name give you a choice on how to explore the site.  The three default choices are shown above.  The operation of these modes can be changed on the "Modes" tab of the options screen.

  You can either press "OK" to do the add the site to the site pane, or "Cancel" to abort.

  Once you add a site to, it will automatically be browsed by SiteSearcher. It will show up alphabetically in the list of top level sites.

• Find files on Page, Find files on Site, and Find files from Links:  These three options specify how SiteSearcher will do it's search on the Site you have currently selected in the site pane.  See the Modes description for information on how these work.  Note that these items may have different names, since their names are configurable in the options window.
• Open...:  This will bring up your default browser, pointing it to the site that you have currently selected in the site pane.  This is useful for looking at a web pages SiteSearcher has found.
• Remove: This item removes the currently listed site from the site pane. Note that any sites that "belong" to this site will also be removed. You can also press "Delete" on the keyboard while you have a site selected to remove it.
• Find in List…: This item brings up the following window:

  

  This window works the same as the File->Find window, with the exception that it will only find one site that contains the phrase you type in. It is not possible to select multiple sites. Note that this will start looking for the phrase starting directly below the currently selected site.

• Cancel Browse: This item will cancel any browsing that is currently taking place. It is useful if you see that SiteSearcher is exploring sites you are not interested in.

View Menu:

This menu contains items related to viewing various areas of the user interface. These items are described below:

• Toolbar: When this item is checked, the toolbar will be visible underneath the menu bar.
• Status Bar: When this item is checked, the status bar will be visible below the two main panes. This bar will keep you informed of important events.
• Downloads...:  This item opens a Windows Explorer window, showing whatever files have been downloaded by SiteSearcher.
• Stats...:  This item brings up a small window that displays some statistics about the number of files found, downloaded, etc.  Note that this window can remain up while SiteSearcher is running, and won't interfere with it's operation.
• Options…: When this item is selected, the following window is displayed:

This window helps you control the many features of SiteSearcher. The window is broken up into sections.  By clicking on the "tabs" at the top of the window, you can view the different sets of options.  When you are done making changes to the options, you can click on "OK" to accept your changes, or "Cancel" to discard your changes.

Note: Any changes you make to the option screen become effective immediately after you hit OK. Any searches that are occurring at the time of the option changes will see the changes.  Be aware of the repercussions that your changes will make.  The exception to this is Mode changes.  Mode changes will not effect any current searches.

File Types Tab
The settings on this tab allow you to specify what type of files to have SiteSearcher search for.  SiteSearcher looks at the file extension to determine what type of file it has found.  The file extension is the last section of the filename, following the period.  For example, the file extension of "FINDSITE.GIF" is "GIF".  If you're interested in looking for picture files, you might add the extensions "JPG", "GIF", and "PCX".  Pressing the "New" button adds a new entry, and allows you to edit it.  If you want to change an entry, select it from the list, and hit "Edit".  If you would like to remove an entry from the list, select it and click the "Delete" button.

File Size Tab

The settings on this tab tell SiteSearcher if and when to check the length of found files.  These settings allow SiteSearcher to filter out files depending if they are too large or too small.  The first button, "Never" turns off file size checking.  The next two buttons, "When file is found", and "When file is downloaded" tell SiteSearcher when to do the check, either when a file is first found, or when the file is downloaded.  If you have this last option set, and you try to download a file that is too large or too small, SiteSearcher will simply ignore the request.

The Min Size and Max Size fields allow you to specify a range of sizes that you would like SiteSearcher to accept.  These sizes are in Kilobytes.

Exclude Tab

The settings on the Exclude Tab allow you to specify which files and sites should be ignored by SiteSearcher.  Any entries in the list will be compared to all files and sites that SiteSearcher finds.  If the file name or the site address contains any of the items in the list, the file or site will be filtered out, and you will never see it in the file pane or the site pane.  Pressing the "New" button adds a new entry, and allows you to edit it.  If you want to change an entry, select it from the list, and hit "Edit".  If you would like to remove an entry from the list, select it and click the "Delete" button.

Misc Tab

This tab contains various miscellaneous settings.  

The Max File Connections entry allows you to specify how many files are downloaded concurrently.  If you are using a modem line, 2 is probably a good setting.  The faster your Internet connection, the higher this number should be.

The Combo History Depth setting specifies how many entries SiteSearcher remembers for certain text entry fields. For example, if this field has a value of 5 in it, SiteSearcher will remember the last 5 text strings you searched for in the "Find Files" window.

The Ask for Site at start up option controls whether or not SiteSearcher asks for a site to search when it is first run.

The Enable Password Logins options tells SiteSearcher whether or not to ask you for a password when searching a secure site.  If you are using SiteSearcher on a site that you need to log into, you should turn this option on.  If you aren't using SiteSearcher to search these kinds of sites, turn this option off.

The Automatically download files check box turns on and off automatic downloads.  This option is only available in the registered version of the program.  When selected, and file that passes all the file filters will be automatically downloaded.  It will not show up in the file pane, but instead be added to the internal list of files to download.

Modes Tab

The settings on this tab allow you to modify the three default Modes that SiteSearcher uses to look for files.  Modes are the most difficult portion of the options to understand.  Most people will probably never need to modify the default modes, since they represent the most common ways SiteSearcher works.  However, if you do have a need to modify a mode, this is the screen to do it on.

A "Mode" tells SiteSearcher what kind, and how many links to follow on each site it finds.  A mode really controls the breadth of the search SiteSearcher does when searching for files.

There is always a default mode.  The default mode is used whenever you double-click on a site to explore it.  It also will be the mode automatically selected when you enter the "Add Site" screen.  To change the default, select the new mode to use as default, and click on the Set As Default button.  "*DEFAULT*" will show up to the right of the item.

The three default modes work as follows:

• Find files on Page:  This will only look on the one page you specify for files.  It will not follow any links, either local or external to the site.
• Find files on Site:  This will only look for files on the site that you specify.  It will follow a large number of links referring to the same site.  It will not look on other sites for files.
• Find files from Links:  This will look for files on the page that you specify, and will also follow a large number of external links to other sites.  Using this mode can result in a search that finds a lot of files, but takes a lot of time.  It is best used with a site that contains a large list of links to sites that have files that you are interested in.

If you would like to modify one of these modes, select it from the list, and click the "Edit" button.  The following window will appear:

The Mode Name setting contains the text that will show up in the various menus and windows that allow you to specify a mode.  This name should be kept a short as possible, in order to nicely fit into menus and windows.

Under Local Sites, you can specify the maximum number of local sites to follow in the Follow Max setting.  If set to 0, SiteSearcher will only search the page you specify, and no other local pages.  If the Follow Parent Sites option is not check-marked, SiteSearcher will only follow links that refer to a sub-page of the current page. For example, if SiteSearcher was on a page with the address of www.aa.bbb/compa, a reference to www.aa.bbb would not be followed. However, a link to www.aa.bbb/compa/divb would be followed. This comes in handy if you want to only search a specific sub-section of a large site.

Under External Sites, there are several settings.  

• Automatically Follow: If this is checked, external sites will automatically be followed. How many are followed are determined by the next two items.
• Follow Max: Specifies the maximum number of external links to follow.
• Max Depth: Specifies the maximum number of "levels" SiteSearcher will go before it stops browsing sites automatically. For example, say you set this to "2". Whenever you start browsing a site, it is considered level 1. So, with this item set 2, it will explore the original site, plus any external sites that the first site references. It will not browse external sites that any of the second level sites reference.  A setting of larger then 2 can result in some huge searches.

Help Menu:

• Contents...:  When this item is selected, your default browser will be loaded, and this document will be displayed.  You can also press <F1> to bring up this documentation.
• Tutorial...:  When this item is selected, a tutorial web page will be displayed, containing several short tutorials on using SiteSearcher.
• About SiteSearcher:  When this item is selected, a small window is displayed, showing various information about SiteSearcher.

Contact Information

Any bug reports, questions, or comments should be e-mailed to:

Zackware@hotmail.com

The home page of SiteSearcher is located at

www.satacoy.com

Distribution of Demo Version

You are hereby licensed to make as many copies of the evaluation version of this software and documentation as you wish; give exact copies of the original evaluation version to anyone; and distribute the evaluation version of the software and documentation in its unmodified form via electronic means. You are specifically prohibited from charging, or requesting donations, for any such copies, however made, without prior written permission of Zackware.

Distribution of Full Version

Distribution of the full release of SiteSearcher is prohibited, except through authorized channels.