CyberSpyder Link Test
Version 2.1.10
Version 2.1.10 fixes two problems
These fixes are:
- When a HEAD request is made, some servers were responding with
a '403 Forbidden' message. A request to the same URL
made with a GET would succeed. The program has been
modified so that if this response is received to a
HEAD request, it will retry the URL with a GET.
- The program has been modified to include the 'Referer' in
requests (the URL of the page containing the URL
being tested. It was found that some servers would
not respond correctly if this field was missing.
Version 2.1.9 fixes two 'bugs' found in Version 2.1.8.
These fixes are:
- The fix made in 2.1.8 re the handling of DNS by a proxy server
caused a bug that would result in the program treating
non-home hosts as if they were part of the home site
causing the program to parse non-home pages for additional
links to follow. This has been corrected.
- A bug was found when a redirect response was received. The result
message recorded includes the URL of the redirected response
and this could occasionally result in a message that was too
long for the database field resulting in a program abort.
Code has been added to avoid this problem.
Version 2.1.8 fixed 'bugs' found in Version 2.1.7 and adds one new feature.
These changes include:
Fix - The format of the request headers sent has been updated to
conform with current practice so they are compatible with
HTTP/1.0 and HTTP/1.1. The format of the request has been
made similar to that used by the latest versions of Netscape
and Internet Explorer.
Fix - The "Proxy to handle DNS" option has been removed. The program
has been modified so that, when a Proxy server is used, the
DNS operation is unconditionally left to be handled by the proxy
server. This should eliminate problems some users have had when
using a proxy server.
Fix - Validation of the host name in a URL has been added to trap malformed
URLs and prevent the program from aborting. A new error message has
been added to identify such errors:
1021 Host name contains an invalid character
(or is missing trailing slash)
Fix - An anchor that uses only a `#` reference was previously reported as
an Error (e.g. ). This has been
reclassified and will now be reported as a Warning. The reporting
has been left in the program to alert users to any cases where the
reference was intended to include a reference to a name anchor.
(The "#" form is sometimes used where a link is required but the
click or mouseover is handled via JavaScript.)
Fix - An anchor that used a name reference containing a `#` character as
part of the name was processed incorrectly and cause the program
to abort. (e.g. a name anchor defined as
and referenced with
cause a program abort). The parsing
of URLs has been corrected to handle this situation.
Fix - Added changes to permit the use of `Starting URLs` that contain commas.
Previously these could not be used as the comma was used when storing
`Starting URLs` to separate the URL from a possible ID and Password.
The problem was corrected by encoding the commas contained in such
URLs to the encoded URL sequence of %2C. The same fix was applied to
deal with the entry of ID/Password URLs.
Fix - Corrected problems dealing with comment areas () that are
longer than 1275 characters. The program will now deal with comment
areas of any length. The same fix was made to script areas
() and the content between select
() tags.
Fix - Added additional validation for hrefs. If the reference
ends with a `#` without the name reference included, an error
is reported (i.e. if the reference is like
). The new error reported is:
3014 HREF content ends '#' but does not
include the 'name' reference
New feature - Modified the `News` button so that, if a test database is
open AND a proxy server has been defined for the test, then the
`News` request will use the defined proxy. This enables users
who must access the Internet via a proxy server to easily get
the latest update information from the CyberSpyder site.
Note that some problems with name anchors previously thought to be bugs are not.
CyberSpyder will continue to report these as errors:
- Name anchors are case sensitive in some browsers. An anchor defined as
and a reference defined as
will not match. While the reference to
"American" will be found by Internet Explorer, both Netscape
and Opera will fail.
- Similarly, an anchor defined as will not be found
by all browsers if a reference is coded as .
The reference is found by Internet Explorer and Netscape, but not
by Opera. Opera also fails to find the item when the reference
is coded to match ().
Version 2.1.7 fixed a number of minor `bugs` and added an optional parameter to
the command line startup to cause CyberSpyder to terminate when it completes a test.
Version 2.1.6 fixed a problem in handling some incorrect server responses to
a "HEAD" request, an error in storing ID/PW information, a problem in in
retrieving pages from some Apache servers, some cell sizing problems in the
Page Size Analysis reports, the handling of the HTML 4.0 " statements (the new feature added to 2.1.1).
Version 2.1.3 fixed all known bugs that were reported for Version 2.1 and 2.1.1
and 2.1.2.
Version 2.1.1 included a couple of enhancements:
- The HTML Page Size Analysis report now uses a table format to improve
readability as well as to reduce the generated file size for
the page.
- And addition option has been added to the Test Options tab of the
setup screen to allow the testing of URLs that appear within