The SRE-http Web Server

version 1.2N


Introduction

SRE-http is a full featured, remotely configurable HTTP (Web) Server for OS/2. Written in REXX, SRE-http is designed to run as filter under GOSERVE, an IBM EWS Internet Server.

SRE-http provides a number of useful features including:

  • Support for multiple hosts
  • Remotely configurable (with complete on-line documentation)
  • Common-log auditing
  • Request-selector-specific and file-specific (HTACCESS) access controls
  • Several types of document redirection are easily specified.
  • A wide variety of cachable server side includes can be dictated by including the well-known NCSA-style elements in your HTML documents
  • Easy to configure support for mappable images and searchable indices
  • Client-specific and browser-specific documents are readily specified
  • Support for server-side-processing using SRE-http addons and CGI-BIN scripts; with numerous tools to help programmers create their own web aware applications.
  • Bundled accessories include a cachable dynamic directory display facilty, a text search facility, and a simple message-box facility.
  • Available addons include a BBS sytem, a discussion-group package, and a directory-indexer/search-engine
  • This document describes how to use SRE-http. While not necessary, the reader should first examine the documentation that comes with GoServe (GOSERVE.DOC). Knowledge of the REXX programming language would also help. However, you can still do a lot of nice stuff with only a rudimentary understanding of GoServe, or of REXX. Lastly, if you don't know HTML, the "markup language" of the world-wide web (WWW), then it's going to be hard to do anything interesting!

    Before reading this manual, we strongly recommend that you peruse the SRE-http users guide -- it's a much easier read!

    Other useful documentation files include:

  • The SRE-http outline describes the steps that SRE-http follows when processing a request.
  • The SRE-http summary provides a general overview of the capabilities of SRE-http.
  • The description of SRE-http parameters gives a detailed description of SRE-http's user configurable parameters.
  • The list of help files summarizes the various help documents supplied with SRE-http.
  • Other GoServe filters.....

    An older alternative to this filter is Don Meyer's GoHTTP HTTP filter system.
    GOHTTP offers CGI-BIN, HTACCESS compatability, common-log auditing, and rudimentary support for server side includes.

    Another alternative is the InterFilt filter of Derek Sims. It has good support for multiple realms and multiple hosts, but lacks server side includes and other features.

    Notes to upgraders

    Prior to version 1.2L, SRE-http was named SRE-Filter . The hope is that SRE-http better communicates the purpose of this ever-growing bundle of software....

    CHANGES.DOC lists the recent changes.



    Table of Contents

  • 1) Installation notes
  • 1.a)Description of files
  • 1.b)A note on SRE-http program structure.
  • 1.c)Variables and Parameters
  • 1.d)Some basic terminology
  • 2)Multiple hosts, logon rights, and other access controls
  • 2.a) Support for multiple hosts
  • 2.b)Providing open access, to selected resources, using PUBLIC_URLS
  • 2.c)Setting logon requirements
  • 2.d)Assigning client privileges
  • 2.e) Restricting access to selected server resources
  • 2.f)Restricting access to directories using HTACCESS files
  • 3)Modifying the request
  • 3.a)Specifying the home page.
  • 3.b)Specifying default documents, and generating directory listings
  • 3.c) Using ~ to specify a home directory
  • 3.d)Establishing virtual directories
  • 3.e)Specifying ALIASES
  • 3.e.i)To modify simple requests (local redirection)
  • 3.e.ii)To implement searchable indices
  • 3.e.iii)To redirect requests for moved documents
  • 3.e.iv)To specify the location of a CGI-BIN script
  • 3.e.v)To indicate a negotiable resource
  • 3.e.vi) Notes about ALIASES
  • 3.f)What to do if the document is not found.
  • 4) Server Side Includes
  • 4.a)Headers and Footers
  • 4.b)SRE-http style server side includes
  • 4.b.i)Dynamic REPLACEments and string REPLACEments
  • 4.b.ii)File INCLUDES
  • 4.b.iii)Using OPTIONs (as incorporated into a request) as string replacements
  • 4.b.iv)INTERPRETing pre-defined REXX-code
  • 4.b.v) SELECTive excludes
  • 4.b.vi) CACHE control
  • 4.c) NCSA HTTPD style server-server side includes
  • 4.c.i)Details on NCSA ssi's
  • 4.d) Order of execution of keyphrases
  • 5) Image Maps, CGI-BIN and SRE-http Addons
  • 5.a) Processing image maps
  • Types of regions
  • 5.b)Creating SRE-http addons
  • 5.b.i)Calling Syntax
  • 5.b.ii)Return Values
  • 5.b.iii)Notes
  • 5.c) Support for CGI-BIN scripts.
  • 5.c.i) CGI-BIN scripts in other directories.
  • 5.c.ii) Notes on creating CGI-BIN scripts.
  • 5.c.iii) Using PERL CGI-BIN scripts.
  • 6) Miscellaneous SRE-http features
  • 6.a) Special directives and special commands
  • 6.a.i)Special directives
  • 6.a.ii)Special commands
  • 6.b)Common-log, browser, and referer audit files
  • 6.c)The RECORD_ALL_FILE running-count
  • 6.c.i)Suppress recording of repetitive requests
  • 6.d) Load balancing
  • 6.e)Creating a user written pre-filter.
  • 6.e.i) Running the GoRemote package
  • 6.f)Creating a user written post-filter
  • 6.f.i)Mailing event specific alerts
  • 6.g) Uploading and deleting files
  • 6.h) Automatic creation of response headers.
  • 6.i) User defined MIME media types
  • 7) Some useful built-in facilities
  • 7.a)DOSEARCH : Search files for a text string
  • 7.a.i)Search Algorithims
  • 7.a.ii)Options
  • 7.a.iii)DOSEARCH.HTM
  • 7.b)SENDFILE : A file transfer facility
  • 7.c)Write to, and read from, message boxes
  • 7.c.i)The message writer
  • 7.c.i)Viewing message boxes
  • 7.c.i)MESSBOX privileges
  • 7.d)The SRE-http procedure library : A macrospace library of useful routines (for the ambitious REXX programmer).
  • 7.e) Client-specific and browser-specific customization
  • 8) Addons: A list of several useful SRE-http addons.

  • Appendix 1: Description of various SRE-http parameter/control files.
  • A1.a)Initialization of parameters file
  • A1.b)Replacement strings file
  • A1.c) Keep track of "hits" file
  • A1.d) Keep track of all requests file
  • A1.d) Keep track of SENDFILE requests file
  • A1.e) SEL-specific access control and permissions file
  • A1.f)Virtual directories file
  • A1.g)Username/password file
  • A1.h) List of aliases file
  • Appendix 2: Summary
  • A2.a)Limitations of SRE-http
  • A2.b)Summary of SRE-http Features
  • Appendix 3: Terminology



  • 1) Installation Notes

    SRE-http consists of several filter program files (SREFILTR.80, SREFMON.CMD, and others), as well as a set of files containing external routines and samples of initialization databases. As noted above, you'll need the GoServe package (ver 2.50 is recommended).

    1. For convenience, let us assume that the GoServe package has been installed in the \GOSERVE directory (your GoServe working directory), and the data directory is set to \WWW.
    2. Copy SREV12L.ZIP to a temporary directory and then unzip it (them) in the usual manner (i.e.; UNZIP SREV12L). From an OS/2 prompt, run INSTALL (a REXX program bundled with SRE-http), and follow the on-line instructions. You will be asked to provide the names of several directories.

      If you select the defaults, INSTALL will create the following directories (you select which drive):


      You will also be given the opportunity to set several SRE-http parameters, and to setup a SUPERUSER. We strongly recommend that you take this opportunity. In particular, setting up a SUPERUSER can help avoid later configuration headaches (though if you are very security conscious, you can skip this without terrible consequence).

      After running INSTALL, you can delete the contents of this temporary directory.

    3. Set up GoServe to run SRE-http:
    4. At this point, the default version of SRE-http is running! You'll probably want to change the values of a number of SRE-http's parameters. Although you can do this by editing the various parameter files (such asINITFILT.80) with a text editor, for the new user it is probably easier to run CONFIGUR.HTM -- a HTML front-end to SRE-http's built-in configurator.

      The most straightforward way of doing this is:


    5. If you need to use a port other then 80, you should:

    Now, all you have to do is create some informative, efficiently organized, and eye-catching HTML documents, paste them together, and add links to the rest of the net. But that's the fun part.

    1.a) Description of the files that comprise SRE-http

    The SRE-http package consists of a number of files. These include the SRE-http REXX programs, the macrospace procedure library, several user-configurable parameter files, samples of "counter and control" files, and some sample HTML files. A complete description of these files is contained in the file list.

    1.a.i)On changing file and directory names

    Many of the directories, and files, mentioned above can have their names changed by modifying variables that are located in INITFILT.80 (the INITFILT_FILE); either by editing the file with a text editor, or by using CONFIG2.HTM. In this documentation we usually refer to files and directories by their "titles" (such as GoServe working directory), but occasionally the "default name" is used (such as \GOSERVE).

    If you change the names of your directories, keep the following in mind:


    1.b) SRE-http structure: multi-threading and macrospace

    To speed up processing, and to facilitate modularization, SRE-http makes extensive use of REXX macrospace.

    Of greater import, SRE-http uses several "helper" threads to handle username lookup, alias matching, SSI-document caching, SEL-specific access control, virtual directory matches, and post-filter processing. These threads are generated upon the first request to SRE-http. In addition, SRE-http stores a number of variables in the local (GoServe) environment.

    For most users, these implementation details are unimportant. However, these threads do use system resources (queues) which remain alive even after GoServe has been turned off. To free these resources, it is recommended that the FREEUP.CMD program be run after GoServe has been killed.
    Citation: SRE-http uses the REXXLIB library to access macrospace and to generate/communicate between threads. A distribution copy of REXXLIB is supplied with SRE-http, and may be used freely with SRE-http. If you wish to use REXXLIB in other REXX programs, contact Quercus Systems for licensing information.
    Lastly, SRE-http monitors the parameter files, including INITFILT.80. Changes to these files will be incorporated into SRE-http's parameters about every 10 seconds.

    1.c) Variables and Parameters

    Throughout this documentation reference is made to variables (aka "parameters") used to change SRE-http's options and features.
    In the interest of thoroughness, this manual discusses the multitude of SRE-http options in terms of directly modifying these parameters. However, most individuals will probably never need to actually edit a file. Instead, almost all of the parameters mentioned herein can be modified using the intermediate-mode configurator (CONFIG2.HTM). In fact, in most cases the simple-mode configurator (CONFIG0.HTM) will do the job -- where appropriate, links to the appropriate section of the simple-mode configurator will be made available for your browsing pleasure .
    If you are a hands-on kind of administrator, feel free to directly modify these parameters. If so, you might also want to read the detailed descripton of INITFILT.80.

    1.d) Some basic terminology

    The SRE-http documentation uses some not-necessarily-intuitively-obvious terms. The following lists a few of the more important ones. For a more complete list, see the terminology appendix.
    URL
    Universal Resource Locator (URL) is a scheme for specifying Internet resources using a single line of printable ASCII characters. A URL should contain a protocol, domain name, port number (optional), the location of the resource, and an (optional) option list (following a ?).
    Examples:
  • http://your.server.net/dir1/sample.htm
  • http://pet.store.com/prices?type=mammals&class=retail
  • Request string
    When a client asks for a URL (say, by clicking on a link), a request string (along with request headers and a request body) is sent to the server (at the domain name and port listed in the URL). The request string consists of three tokens: the http method, the location of the resource on the server, and the http protocol.
    Examples (assuming the above URLs):
  • GET /dir1/sample.htm HTTP/1.0
  • GET /prices?type=mammal&class=retail HTTP/1.0
  • Selector (also referred to as the request selector )
    The selector is the location of the resource on the server. Specifically, it's the (slightly modified) second term in the request string.
    Examples (assuming the above request strings):
  • dir1/sample.htm
  • prices?type=mammal&class=retail
  • Note that the "slight modifications" consist of:
  • dropping the leading /
  • decoding URL-encoded characters (note that the above examples do not include any URL-encoded characters).
  • In many cases, the connection between the selector and server resource it refers to is simple (such as when the selector is a filename relative to the GoServe data directory). In other cases, it's completely virtual (as when the resource consists of the output of a program that's run using information provided by the client).

    Selector-specific (usually written as SEL-specific )
    SEL-specific refers to SRE-http configuration information that is specific to a selector -- such as resource privileges, or a virtual directory.
    The selector is a pointer to a web-server resource. Typically, this would be a file; but it might refer to the output of some program run just for this request. In either case, SRE-http has to match the selector to the actual resource (an activity that can become rather convoluted!).
    In much the same way that OS/2 extended attributes contain additional information about a file, SRE-http's SEL-specific information contain additional information about a selector (needless to say, this information is only used by SRE-http).

    Return to table of contents




    2) Multiple hosts, logon rights, and other access controls

    A summary of this section:

    2.a) Multiple hosts

    To use SRE-http in a multi-host environment, you must specify host-specific entries that assign a data-directory and a host-nickname to a specific IP address. You should also include a host-nickname when defining a number of SRE-http parameters.
    You can use the simple-mode configurator to create these host-specific entries.

    The HOSTS. stem variable (set in INITFILT.80) defines the hosts (either IP addresses or aliases) that your server will work with. Each HOST. variable should contain the following information (in a comma delimited list):

     HOSTS.n ='ip_address, host_nickname,default_dir' 
    Where:
  • ip_address is a IP address (integer or numeric)
  • host_nickname is the "nickname" you will use when referring to this host. The host_nickname must not be a number, it must contain at least one non-digit character. Thus, a host nickname of 1234 is invalid, but 1234X is okay.
  • default_dir is the default directory for this host (it is used instead of the GoServe default data directory).
  • If your server recieves request from a host which does not have a matching HOSTS. entry, it will be treated as "generic" request; that is, the standard (single host style) default directory, etc. will be used with such requests.
    Thus, if all requests (regardless of intended host) have identical access to your server's resources (it's files and programs) you can ignore this entire section!

    Given that you do want to have different resources available to different hosts, almost all of the SRE-http variables and parameters can be specified on a "host specific" basis

    Notes



    2.b) PUBLIC_URLS

    There may be cases where you generally do not want to allow public access to your site, except for a set of clearly delimited public areas. For example, you may have a private site, except for a set of "who we are" documents. Although one could use selective access controls to accomplish this, the use of PUBLIC_URLS provides a simple and direct alternative.
    You can use the simple-mode configurator to create these PUBLIC_URLSs.
    The PUBLIC_URLS. stem variable is used to identify server resources that are "open to the public" -- that can be accessed by clients who have no logon rights and no access privileges.

    Basically, SRE-http compares the request (using wildcard matching with substitution) against the entries in the PUBLIC_URLS. stem variable.

    All requests that match a PUBLIC_URLS stem variable:


    PUBLIC_URLS can also specify literal public_urls. When a request selector matches one of these literal public_urls; file transfer is expedited by: More importantly, the SREFQUIK variant of SRE-http is written to quickly transfer these literal public_urls-- cutting response time by as much as 50% under moderate to heavy load

    Notes:


    2.c) Logon Controls

    The simple-mode configurator can be used to enable LOGON, to add users to the USER_FILE, to add/remove IN-HOUSE clients, and to enable the LOGON_FAIL_FILE.

    Setting LOGON requirements with the CHECKLOG variable

    CHECKLOG should have one of the following values:
    NO
    No logon required; unencumbered entry for all clients.
    YES
    Logon upon invocation of the default page (i.e.; an empty request).
    ALWAYS
    Logon required for all requests.
    INHOUSE
    Entry allowed only to clients with INHOUSE or SUPERUSER privileges.

    Allowing unencumbered access to IN-HOUSE users

    The INHOUSEIPS. stem variable can be used to specify the numeric IP addresses of clients who are to be given "IN-HOUSE" status. IN-HOUSE client do not have to enter logon information, and they are automatically granted the privileges stored in the INHOUSE_PRIVS variable.
    Notes

    Owners have SUPERUSER status

    You can give SUPERUSER privileges to a set of numeric IP addresses by changing the value of the OWNERS variable.
    Notes:
  • Wildcards are not allowed in the OWNERS variable
  • OWNERS are given INHOUSE_PRIVS privileges, as well as SUPERUSER privileges.
  • The OWNERS variable must be host-specific (see INITFILT.DOC for details).
  • Logon for non IN-HOUSE users.

    If a logon is required, an authorization string (containing a name and password) is obtained from the client. SRE-http then examines the USER_FILE for a match. If a match is found, the request is honored (abstracting from SEL-specific or directory-specific access controls). Otherwise the client is asked to re-enter her name and password. Note that entries in the USER_FILE may be "host specific".

    Be aware that ...

    Keeping out known pests

    There may be cases where one needs to deny access to particular client(s). This can be accomplished by recording the numeric IP addresses of these "unwanted clients" in the UNALLOWEDIPS.n stem variables (wildcards are permitted).

    Note that the OWNERS and INHOUSEIPS.n stem variables are checked first. If the client's numeric IP address appears in either of these lists, UNALLOWEDIPS.n is not checked. Thus, by setting UNALLOWEDIPS.1="*", you can deny entry to everyone except OWNERS and IN-HOUSE users.
    Also note that UNALLOWEDIPS. entries must be "host-specific" (see INITFILT.DOC for details).

    Detecting fake IP addresses

    For additional security, you can set the DNS_CHECK variable (to YES). When set to YES, SRE-http will lookup the client's IP name; and if it does not exist (there is no DNS entry), logon will be denied. This provdes some security against "spoofers", but it also slows down response time, and may keep out legitimate users using dynamic (and possibly unnamed) numeric IP addresses.

    The LOGON_FAIL_FILE

    If the client does not have "logon rights", SRE-http will use the value of the LOGON_LIMITand the LOGON_FAIL_FILE to determine the response.
    For further details, see the description of LOGON_LIMIT and LOGON_FAIL_FILE in INITFILT.DOC.

    2.d) Assigning client privileges

    In several circumstances, SRE-http will check the "client privileges" before transferring information. In particular, file transfer when SEL-specific access controls are in place (i.e.; when ALLOW_ACCESS='NO') requires that the client have appropriate privileges.
    In a sense, privileges offer a convenient shorthand for identifying sets of users.
    Privileges are set automatically for:
  • IN-HOUSE clients are granted privileges stored in their INHOUSEIPS. variable,
  • OWNERS are granted SUPERUSER privileges.
  • In addition, IN-HOUSE clients and OWNERS will be granted the privileges that are listed in the INHOUSE_PRIVS "space delimited list" variable. It is recommended that INHOUSE_PRIVS always include the INHOUSE privilege.
    For everyone else:
  • If a logon is required, the privileges are read from the USER_FILE.
  • Last of all ..
  • Privileges listed in the the PUBLIC_PRIVS "spaced delimited list" variable are given to everyone.
  • Special Feature: Dynamic privileges

    In addition to the above "static" privileges, clients can be assigned "dynamic" privileges. These dynamic privileges are typically set by prior actions (say, filling out a registration form, or requesting an introductory file), and have a short life expectancy (measured in minutes). For further discussion of these "dynamic" privileges, see ADDPRIVS.DOC, and the discussion of the CHECK_ADD_PRIVS and ADD_PRIVS_PREFIX variables in INITFILT.DOC.

    2.e) Accesss control on a selector-specific basis.

    The simple-mode configurator can be used to enable selector-specific access controls, and to add/remove selector-specific access control entries
    There may be cases where you want clients to have unencumbered access to some, but not to all, the resources of your server. One way to do this is to identify which request selectors a client has access to. The ACCESS_FILE, and the ALLOW_ACCESS variable, are used to construct these selector-specific (SEL-specific) access controls.

    ALLOW_ACCESS can take the following values:

    YES
    No limits -- all clients (with logon rights) are automatically given access rights to the server resource.
    INHOUSE
    IN-HOUSE users (and SUPERUSERs) are given automatic access rights, all others must have appropriate privileges.
    NO
    No one (except SUPERUSERs) is given automatic access rights, all others must have appropriate privileges.
    When ALLOW_ACCESS is enabled, SRE-http will examine the ACCESS_FILE for an entry that matches the request selector .

    Additional Features of the Access Control File

    In addition to general access rights (do you have any rights to the server resource), the ACCESS_FILE contains several additional pieces of SEL-specific information:

    Permissions
    The permissions list includes information on:

    The Realm
    When ALLOW_ACCESS is enabled, and the client does not have appropriate privileges, SRE-http will query the client for a new username/password. When querying, the SEL-specific realm is displayed (if available). If there is no SEL-specific realm, the THE_REALM realm is displayed.

    ACCESS_FAIL_FILE
    If the client does not have access to the server resources represented by the selector, SRE-http will use the value of ACCESS_FAIL_FILE to dictate the response. If ACCESS_FAIL_FILE=0, a username/password will be re-requested. Otherwise, the ACCESS_FAIL_FILE, or a SEL-specific access failure file, will be returned. For details, see the discusion of ACCESS_FAIL_FILE in INITFILT.DOC.

    Advanced Options
    The SEL-specific advanced options file allows you to specify several SEL-specific advanced options, including:


    Notes: Two types of resource privileges

    There are two types of resource privileges :
    ONE_OF (the default type).
    The client must have "one of" the ONE_OF privileges.
    MUST_HAVE (start with the & character)
    The client "must have" all the MUST_HAVE privileges.
    Note: if only one privilege is specified, these two types are equivalent.

    Notes: Matching client privileges to resource privileges

    SRE-http's privilege matching algorithim is quite simple:
    1. If a MUST_HAVE privilege appears in the SEL-specific resource privileges list, then it must also appear in the client privileges list. If not, access is denied.
    2. After checking all of the MUST_HAVE privileges, SRE-http then compares each ONE_OF privilege to the client privilege list. If any of these ONE_OF privileges appear, then access is permitted. Otherwise, access is denied.
    3. Reiterating: access is denied if there is no overlap between the client's privileges and the ONE_OF resource privileges

    Notes: SEL-specific permissions

    SEL-specific permissions are used to provide SEL-specific control of the features of SRE-http. That is:
  • SEL-specific privileges are used to control access to a resource
  • SEL-specific permissions are used to control SRE-http features, assuming that the SEL-specific privileges have been satisfied!

  • Currently, the following permissions are available:
  • Suppressing server-side actions You can suppress server-side-includes, and server-side-processing on a "SEL-specific" basis by including NO_SSI and NO_SSP permissions (respectively). You can also suppress "in-document" server-side processing (the SELECT and INTERPRET CODE keyphrases) by including a NO_CODE permission (note that NO_SSP implies NO_CODE).

  • Permitting the PUT and DELETE methods SRE-http understands the PUT and DELETE HTTP methods. The PUT method is used to upload a file to your server, and save it using the location specified in the request. The DELETE method is used to delete a file, as specified by the location specified in the request.

    For security reasons, PUT and DELETE requests will only be honored if the appropriate permssion (PUT or DELETE respectively) appears in the permissions list .
    Furthermore, when using the PUT_FILE or GET_URL facilities to upload to a non-upload_directory subdirectory (that is, to a virtual directory), the PUT permission must apply (to the "selector" that matched the virtual directory).

  • Suppressing examination of HTACCESS files To suppress the use of HTACCESS files for selected server resources, include a NO_HTACCESS permssion. Note that this will not effect the SHOWDIR dynamic directory list processor.

  • Suppressing virtual directory, post-filter, and aliasing The NO_VIRTUAL, NO_POSTFILTER, and NO_ALIAS permissions will suppress virtual directory lookup, post-filtering and file recording, and alias lookup (respectively) on a SEL-specific basis.

  • Allowing a file to be cached Whenever there is a possiblity that access to a resource will be denied to a client, SRE-http will disable caching of the file invoked by the request selector . In particular, if CHECKLOG is not equal to "NO", or if ALLOW_ACCESS is not equal to "YES", then caching will be suppressed.
    While an important security feature, there may be cases where you are willing to sacrifice some access control in order to speed up request processing. To achieve this, you can enable caching (on a URL specific basis) by including CACHE in the permission list.
    Note that for caching to occur, the GoServe caching option must be turned on!
  • Notes: Miscellaneous

    Return to table of contents


    2.f) Directory-specific access control, using HTACCESS files.

    To provide compatability with HTTPD style servers (such as Don Meyer's GOHTTP), SRE-http supports the HTACCESS method of access control. This support uses code graciously loaned to us by Don Meyer, suitably modifed for SRE-http.

    Briefly, the HTACCESS method utilizes special HTACCESS files that are placed in your various directories (usually with the name HTACCESS., or .HTACCESS). These files (there may one in each of several directories) contain entries pointing to password files, user-group files, and other parameters that are used to ascertain who shall be given access to the files in the directory (and it's child directories).

    HTACCESS control will be checked for all requests. Note that the presumed location of the HTACCESS file depends on the type of request:

    In all these cases, the "parent" directories are also checked for HTACCESS files.

    Notes on the use of the HTACCESS method





    3) Modifying the request

    URL's can get long and nasty, documents are often moved, people often mistpye URLs or don't ask for an exact filename, and sometimes it's just nice to use shorthands when specifying the location of your server resources. These actions are examples of modifications to the request.

    This section discusses ways you can conveniently modify & redirect a request (as recieved from the client).


    3.a) Specifying the home page (the default)

    When the client sends an empty request, a /, or "/?some+options", SRE-http assumes that the client desires the "home page" (the default document). By tradition (but not necessarily) the default document is a file named INDEX.HTM located in the root of the data directory.

    The default can be set using the DEFAULT or DEFAULT.HOST_NICKNAME variable (that is, DEFAULT may be host specific).

    You can use the simple-mode configurator to set the home page and the directory-specific default document(s)

    Return to table of contents


    3.b) Specifying default documents for directories

    Often, a request may arrive that specifies a directory, but does not specify a file. For example, a request for COOLJUNK/JOKES/ does not specify a particular file, but instead implies "give me the default document for the COOLJUNK/JOKES subdirectory of the data directory". In these cases, the AUTO_NAME variable is examined.

    AUTO_NAME should contain a space delimited list of filenames to use when a request for a "directory" is recieved. SRE-http will attempt to find these files in this "requested" directory -- starting with the first name in the AUTO_NAME list. To assist in this search, a special name, consisting of *.HTM (or *.HTML) means use the directory_name.htm file.

    Lastly, an entry of !CREATE instructs SRE-http to generate a directory listing of files and subdirectories in the requested directory. This listing is generated with SRE-http's !DIR special action. By default, a <PRE> formatted table is generated, containing (for each file):

    If you are interested, it is fairly easy to modify these display options -- you can even tell SRE-http to automatically generate a description for each file.

    Example

    AUTO_NAME=" *.HTM INDEX.HTM "
    If the request is for /INFO/MAINE/, and your data directory is D:\WWW, then ...
  • *.HTM is interpreted as: D:\WWW\INFO\MAINE\MAINE.HTM
  • INDEX.HTM is interpreted as: D:\WWW\INFO\MAINE\INDEX.HTM
  • Notes


    Return to table of contents

    3.c) Using ~ to specify the home directory

    You can use the simple-mode configurator to set the home-directory
    As a convenience, SRE-http will replace occurences of ~ characters in request selector with a string. Since the ~ is often used to point to a set of individual "home directories", this replacement string is stored in the HOME_DIR variable.

    To select the home directory , change the value of the HOME_DIR (or HOME_DIR.HOST_NICKNAME) variable.

    Example
    If ...
  • HOME_DIR="BRANCH/STAFF/PERSONAL/"
  • and
  • The request is ~JOES/PAGE.HTM,
  • .. then
  • SRE-http will look for: BRANCH/STAFF/PERSONAL/JOES/PAGE.HTM (under your data directory, or in a virtual directory).
  • Notes

    Advanced Option: Specifying User Subdirectories

    In many cases, you may wish clients to limit access to particular subdirectories of your HOME_DIR directory. For example, suppose all "students" have space on the HOME_DIR directory of your web server; or HOME_DIR may be on a LAN drive accessible as a virtual directory from the web server. These "student" directories may contain several subdirectories; one of which is intended for WWW use, while the others are for personal use.
    The goal is to give clients direct access to the "web" related directories but not to the "personal" directories.
    This goal can be achieved by including a $ in the HOME_DIR parameter. Specifically, the $ is replaced by the portion of the request between the ~ and the first / following this ~

    For example: If:

    Then SRE-http will use: /USERS/GERALD/WWW/RESUME.HTM

    For futher details, see the discusssion of the HOME_DIR parameter in INITFILT.DOC.


    3.d) Establishing virtual directories

    You can use the simple-mode configurator to add/remove virtual directories.
    By default, SRE-http will match the request selector to a file in the data directory; or if the request is for server side processing (such as an SRE-http addon, or a CGI-BIN script), to a file in the addon directory or CGI-BIN directory. While a good security feature (files not in these directories are inaccessible), this can be an inconvenience. To remedy this inconvenience, one can define "virtual directories" using the VIRTUAL_FILE. Basically, SRE-http will compare the starting portion of a URL to see if it matches an entry in VIRTUAL_FILE. If it does, the directory listed in the entry is used (instead of the data directory or addon directory).

    Thus, you can make available a wide, but controllable, set of "local" directories (on or LAN accessible from) your server.

    In addition, "virtual directories" can point to "remote" directories on other http servers -- SRE-http will attempt to retrieve the file from the remote server; without using a redirection.

    To clarify: to the client, remote virtual directories are indistinguishable from "local" virtual directories.

    In particular, HTML files with server side includes can be remotely located -- and if the remote server makes no attempt to process these server side includes, SRE-http will!

    For details on the use of the virtual directories, see the description of the VIRTUAL_FILE .

    Note that entries in the VIRTUAL_FILE may be "host specific"


    Return to table of contents


    3.e) Using "aliases" when responding to requests

    ALIASES are used in SRE-http to redefine and redirect requests
    Along with the ALIAS_FILE, the CHECK_ALIAS parameter controls whether and when ALIAS checking occurs.

    CHECK_ALIAS should have one of the following values:

  • NO = Suppress ALIAS checking
  • YES = Enable ALIAS checking
  • Major uses for ALIASES

    3.e.i) Local redirection

    3.e.ii) Processing searchable indices

    3.e.iii)Remote redirection

    3.e.iv) Redirecting requests for CGI-BIN scripts

    3.e.v) Content Negotiation

    SRE-http supports both server-side (http/1.0) and client-side (http/1.1) content negotiation. In either case, you use a special ALIAS to indicate that a given selector is "negotiable".

    The basic syntax of this alias is:
       sel !NEGOTIATE
    where sel is a selector, and !NEGOTIATE indicates that sel is a "negotiable" resource.
    For example: VARTEST/VAR3.VV !NEGOTIATE

    Actually, sel should be a "variant list" -- SRE-http uses "variant lists" to determine which "alternate" (of several possible variants) to use.
    Alternatively,you can use:
       sel !NEGOTIATE variant_file
    where sel is a * wildcard containing selector, and variant_file is a valid selector pointing to the variant file.
    For example: INFO/* !NEGOTIATE INFO/MANUALS.LST
    INFO/MANUALS.LST would be the "variant file" -- it would be used for all request selectors that match INFO/*. If you use this "wildcard" construct, be sure to include an appropriate PATTERN: in the variant file.
    For details on how SRE-http's support for content negotiation, see NEGOTIAT.DOC.

    3.e.vi) Notes about Aliases

    Return to table of contents


    3.f) What to do if no document can be found

    Sometimes, despite the use of "aliases" and "AUTO_NAME", the requested "action" (such as an HTML document, or a CGI-BIN script) can not be found. In those cases, a 404 Not found response is returned to the client, along with a short description. You can put a customized message in this short description. In particular, you can put a link to your "home page".

    The NOT_FOUND_URL variable contains this customized message. By default, it's value is:
    <a href="/"> Visit the HOME_NAME home page? </a>
    Note that HOME_NAME will be replaced by the current (possibly host specific) value of the HOME_NAME variable.
    Thus, the befuddled client merely needs to click on this link to get some kind of access.

    Notes:

    Return to table of contents



    4) Using Server Side Includes

    One of the primary advantages of SRE-http is the extensive support for a variety of
    server side includes. These server side includes (SSI's) let you easily create dynamic documents. In particular, inclusion of files, text strings, and transient variables (such as the time, date, and number of requests) are easy to accomplish, requiring modifications to your HTML documents, and not to the server software.

    SRE-http supports two sets of SSIs -- the NCSA set of SSIs, and a set of SRE-http-style SSIs. Both sets require the inclusion of special keyphrases in one's HTML documents. When an HTML document is requested, SRE-http will (optionally) detect these keyphrases, and perform some sort of textual substitusion.

    The basic structure of a keyphrase is:

  • <!-- KEYWORD argument -->
  • where: Examples:
  • <-- REPLACE CREATION_DATE --> :
  • <-- INCLUDE ADDLIST.HTM -- >
  • <-- #FLASTMOD VIRTUAL="SAMPLES/srehttp.htm" -->
  • Alternate keyphrases

    In the interests of brevity, and to help avoid conflicts with other html comments that aren't comments , there are several modifications you can make to these KEYWORDS.
    1. If desired, you can add a "preface" (such as the ! character) to the definitions of the first six keywords (i.e.; !REPLACE is used instead of REPLACE). For further details, see the description in INITFILT.DOC of the KEY_PREFACE variable.
    2. You can use several shorthands: For further details, see the descriptions in INITFILT.DOC of the QUICK_INTERPRET, QUICK_INTERPRET_CODE, and QUICK_REPLACE variables.

    Miscellaneous Notes


    4.a) Headers and Footers

    In addition to server side includes using keyphrases, a multi-line header and footer can be automatically added to all HTML documents. The header is added just after the (first) <BODY > tag in the document (see the notes for an exception to this rule); and the footer is added just before the (last) </BODY > tag.

    To set these, just change the values of the HEADERS. (for headers) and the FOOTERS. (for footers) stem variables located in INITFILT.80. Or, you can use the simple-mode configurator.


    Header and Footer Notes

    Return to table of contents


    4.b) SRE-http style server side includes

    4.b.i) The REPLACE keyphrase

    Syntax: <!-- REPLACE Varname --> , where Varname is the name of a static or dynamic variable. Note that you can use % instead of REPLACE.

    The REPLACE keyword is used to insert dynamically determined information and short (static) strings into a document. SRE-http recognizes the following "built-in" Varnames (the underlined variables are user-configurable, possibly host-specific, "static" variables that can be modified by editing INITFILT.80): :

    A special argument can be used to extract variables from the GoServe environment, which // contains many variables created by SRE-http.

    Last of all, if none of the above matches argument , the custom REPLACEments are searched for a match (which might be host specific). If a match is found, the information from this custom REPLACEment is used.
    You can even include argument placeholders to this custom replacement; they will be replaced by arguments included in the REPLACE keyphrase.
    To create custom REPLACEments, you should edit the REPSTRGS.IN file (or you can use the simple-mode configurator).

    Examples
    <!-- REPLACE TIME -->
    <!-- % URL -->
    <!-- % ODOM MYDOC1 -->
    Notes

    Return to table of contents


    4.b.ii) The INCLUDE keyphrase

    Syntax: <!-- INCLUDE FileName --> , where FileName is the name of the File to be included.

    The INCLUDE keyphrase instructs SRE-http to include the contents of a file into the requested document (where argument is the file-to-be-included). For example, INCLUDE PHONE.LST would cause the contents of the file PHONE.LST (in your data directory) to be inserted into the document.

    Examples
    <!-- include buttons.bar -->
    <!-- include newtoday.txt -->

    Special Feature: Browser and Client Specific File Includes
    You can use the ByBROWSR.RXX INTERPRETable utility to selectively include browser-specific files into your documents. These can be complete documents (with the "requested document" containing only a stub); or you can include smaller browser specfic components in an otherwise all purpose document.

    In a similar vein, the ByCLIENT.RXX INTERPRETable utility can be used to selectively include files on a "client-specific" basis, where clients can be identified with cookies, privileges, or their IP address.

    Notes

    Return to table of contents

    4.b.iii) The OPTION keyphrase

    Syntax: <!-- OPTION n --> , where n is an integer.

    The OPTION keyphrase instructs SRE-http to insert the contents of the n'th OPTION into the document.
    OPTIONs are contained in the portion of the request following the ? character.

    How to pass OPTIONs to a document
    To pass OPTIONs to a document, the selector portion of the request should have the form:
    HTMLFILE.HTM?Your+OPTION+1&OPTION2

    OPTIONs and searchable indices.
    When a browser responds to document containing the <ISINDEX > element, the request will have the same HTMLFILE.HTM?word1+word2 format. In order to properly respond to such "searchable index generated" responses, you should set up an ALIAS
    Examples
    <!-- Option 1 --> <!-- Option 3 -->
    Notes


    Return to table of contents


    4.b.iv) The INTERPRET keyphrase

    Typical syntax: <!-- INTERPRET FILE xxx.RXX -->

    INTERPRET provides a means of including and executing a REXX-code-block

    There are two major variants of the INTERPRET keyphrase:

    1. INTERPRET CODE statment 1 ; statement 2 ; ... ; where each statement n is a valid REXX statements (and each statement is seperated by a semi-colon). For example:
      INTERPRET CODE tmp1=server() ; interpret.result=tmp1
      A Shorthand: You can use # instead of INTERPRET CODE
    2. INTERPRET FILE Filename arg_list , where:
    3. Filename is a file containing REXX code (filename is assumed to be relative to the SRE-http addon directory, or to a "local" virtual directory).
    4. arg_list is an optional comma delimited list of "arguments"
    5. A Shorthand: You can use $ instead of INTERPRET FILE
    Example
    If your document contains the keyphrase:
    <!-- INTERPRET FILE GOBED.RXX-->
    ( or, equivalently, <!-- $ GOBED.RXX--> )
    and GOBED.RXX contains:
    tmp1=time('h') ; if tmp1<6 | tmp1>23 then interpret.results=" <em> It is late, GO TO BED! </em> " ;
    then the phrase It is late, GO TO BED! will appear in your document whenever your local time is between 11:00 P.M and 6 A.M.

    In general, the INTERPRETed REXX-code-block should include one or more of the following:

    1. A SAY or AUDIT command (for writing status messages)
    2. Assignment of a (possibly multi-line) string to the INTERPRET.RESULTS variable. The contents of INTERPRET.RESULTS is written into the document (at the location the keyphrase occupies).
    3. A QUEUE (or PUSH) command -- with each QUEUEd (or PUSHed) string automatically appended to INTERPRET.RESULTS (see the discussion of the USE_STDOUT variable in INITFILT.DOC for more details on this #EXEC-like feature)
    4. RETURN 'stuff to include'
    Notes:


    Return to table of contents


    4.b.v) The SELECT keyphrase

    The SELECT keyphrase is qualitatively different from the other keyphrases -- it acts as a "conditional server side exclude" of HTML code that is already in your document. The power of SELECT is the ability to check dynamic variables (such as the client's IP address, or his user agent), and retain or exclude a portion of the HTML document accordingly.

    A SELECT keyphrase expects an argument containing a block of REXX code, just like the code blocks used in the INTERPRET keyphrase. This block of code will be interpreted. The code block should "exit" with either a RETURN 1 or RETURN 0 , where:

  • RETURN 1 : the subsequent HTML text block is retained
  • RETURN 0 :the subsequent HTML text block is deleted.
  • The "subsequent HTML text block" is delimited by a SELECT END keyphrase, and can be of any length (if no SELECT END keyphrase is encountered, the remainder of the file is dropped). Lastly, if there is no RETURN, the code block is retained (i.e.; the default is RETURN 1).

    Example:

           <!--SELECT  tmp.1=extract('clientaddr') ;
           if wordpos(tmp.1,owners)=0 then return 0; -->
           <em> Hello!  You have SuperUser Privileges !! </em>
           <!-- SELECT END -->
    
    Note that RETURN 1 is assumed by default, hence when the clients IP address is listed in the an OWNER variable (and the code block ends without a return) the Hello! ... message will not be deleted.

    Notes


    Return to table of contents


    4.b.vi) The CACHE keyphrase

    You can use the simple-mode configurator to enable caching of server side includes
    To improve performance, SRE-http will "cache" HTML documents that have server side includes (SSI). This caching is of the document after the SSIs have been performed. In simple cases SRE-http can send this cached file, without having to repeat the actual process of SSI lookups, etc. Needless to say, this can greatly improve server performance.

    For a complete description of SRE-http's SSI-caching algorithim, and of the various options available for the CACHE keyphrase, please read SSICACHE.HTM . What follows is a very brief description of the options available for the CACHE keyphrase.

    <!-- cache no -->
    Do not cache this document
    <!-- cache asis -->
    Use a cached version "as is"
    <!-- cache notrigger -->
    Do not check the trigger files; but do check the own file.
    <!-- cache duration nnn -->
    Set the lifespan for this file's cache entry, where nnn is the number of (possibly fractional) days.
    <!-- cache trigger file1 file2 .. -->
    Check the file-stamps of these fully qualified files.
    <!-- cache trigger_type stamp -->
    Selects which field(s) of the trigger file-stamps to check.


    Return to table of contents


    4.c) Support for NCSA HTTPD server side include

    With only minor exceptions, the server side include syntax of the NCSA HTTPD server is fully supported by SRE-http. These are invoked with a keyphrase of:
    <!-- #cmd option="value" -->
    Where cmd can be: INCLUDE CONFIG FLASTMOD FSIZE ECHO EXEC

    4.c.i) Details on NCSA HTTPD server side includes

  • EXEC: Execute a CGI script.
    option should equal CMD, and value should be a file name (in the CGI_BIN_DIR directory) of a CGI script (typically a REXX program or an executable). Note that the EXEC server side include captures the output of the script, and includes this output in the requested document. EXEC will not be done if NO_PROCESSING=YES, or if a NO_SSP permission applies to this selector.
  • ECHO: Value of a variable
    option should be VAR, value should be a variable name. This is equivalent to the REPLACE keyphrase, but with different variable names (although many of them overlap).

    The supported variables are:

    DOCUMENT_NAME, DOCUMENT_URI, DATE_LOCAL, DATE_GMT, LAST_MODIFIED, SERVER_SOFTWARE, SERVER_NAME, GATEWAY_INTERFACE, SERVER_PROTOCOL, SERVER_PORT, REQUEST_METHOD, SCRIPT_NAME, QUERY_STRING, REMOTE_HOST, REMOTE_ADDR, AUTH_TYPE, AUTH_NAME, HTTP_ACCEPT, HTTP_REFERER, HTTP_USER-AGENT
    Unsupported variables are (they really aren't relevant to server side includes):
    PATH_INFO, PATH_TRANSLATED, REMOTE_IDENT, CONTENT_TYPE, CONTENT_LENGTH
    Need more information on NCSA HTTPD server side includes?


    Return to table of contents


    4.d) Order of execution of SSI keyphrases

    Server-side includes are processed in the following order:
    1. The requested document is read into memory.
    2. Headers and Footers, if specified, are included.
    3. The document, (that now includes headers and footers), is scanned for keyphrases.
    4. When a keyphrase is encountered, the appropriate server side include is performed; or if a SELECT keyphrase is encountered, a server side "exclude" may be conducted.
    5. After each keyphrase is processed, step 4 is repeated (from the beginning of the file).
    6. This process continues until there are no more keyphrases.
    7. If you've specified more then one set of keyphrase delimiters, step 3 will be repeated for each set.
    Note that the keyphrases in the latter part of a document are processed after freshly included keyphrases. While not the most efficient algorithim (long documents with many server side includes will cause a lot of string comparisons), it does enable recursive processing of all keyphrases

    This recursive feature allows for some powerful document manipulation strictly through the use of keyphrases. In particular, the INTERPRET keyphrase can examine variables such as the "referer", the "client name", or the "OPTION list", and generate inclusions that incorporate other keyphrases (which will be processed on the next cycle of keyphrase interpretation).

    There are several caveats:


    Return to table of contents


    5) Imagemaps, CGI-Bin Scripts, and SRE-http addons


    The modern web site is apt to provide an array of service to it's clients. These services, which imply more then just the delivery of documents and images, are based on an interaction between the client and the server; with the server using information recieved from the client, recording and otherwise manipulating this data, and returning some results.

    Delivery of this server side processing typically requires installation of some sort of program that adheres to an application programming interface (API) for communicating to web clients via a web server. SRE-http supports several such API's, including:

  • NCSA and CERN server side imagemaps
  • GoServe/SRE-http addons API
  • The Common Gateway Interface (CGI-BIN)

  • 5.a) Using imagemaps

    SRE-http provides a simple means of responding to image maps, using NCSA or CERN style "MAP" descriptor files. The key is to include a map-image indicator substring in the the "resource location" of a URL that points to a .MAP file.

    More specifically, when a MAPIMAGE/ substring is encountered in the action portion of the request, SRE-http will remove MAPIMAGE/, and assume the remaining portion of the action refers to a NSCA MAP file (similarly, a MAPCERN/ substring refers to a CERN MAP file).

    For example, if the request selector is:

    MAPIMAGE/PROJECTS/BRANCH1.MAP?55,123
    then the "mappable image" handler would look for PROJECTS\BRANCH1.MAP (under the data directory, or a "local" virtual directory), and use the instructions therein (in conjunction with the pixel location 55,123) to "redirect" the client.

    5.a.i) Type of regions recognized.

    There are four types of regions recognized by SRE-http:
    1. Circles
    2. Rectangles
    3. Polygons
    4. Points
    5. and you should always specify a default URL
    The selected pixel will be assigned to the URL associated with the first Circle, Rectangle, Polygon, or 'exactly-hit' Point it matches. If there is no such match, and there are Points specified in the MAP file, then the URL associated with the closest Point is used.
    However, if the distance to this closest Point is greater then the value of the MAX_POINTDIST variable, or if there are no Points specified in the MAP file, the default URL is used.

    Notes:

    Please remember that the MAPIMAGE/ "indicator string" does not refer to the MAPIMAGE/ directory!

    Return to table of contents


    5.b)Creating SRE-http addons

    As an alternative to CGI-BIN scripts, SRE-http addons can provide a faster (and less convoluted) REXX based method of creating web aware applications. The basic idea is to write an "external" REXX procedure which will do the processing, or which calls some other executable program (such as a database manager) -- and then tell SRE-http to call this procedure.

    The following steps outline how to do this:

    1. Create an HTML document that returns an action that invokes a Server Side Processing request. For example, you might include a FORM element with an ACTION attribute of "/XXX".
    2. Create a file (that contains your REXX procedure), with the name xxx.CMD (or xxx.80) , where xxx is the action name. Note that the xxx file is relative to the ADDON directory (or a "local" virtual directory).
      For example, if the action name is /PROGS/FAST2, and \GOSERVE\ADDON is your addon directory, SRE-http will look for \GOSERVE\PROGS\FAST2.CMD.
    3. This REXX procedure should process the request (using information passed to it by SRE-http), and it may call other programs (not necessarily REXX programs).
    4. xxx should issue GoServe completion codes (such as VAR, FILE, and STRING).

    5.b.i) Calling Syntax

    SRE-http will call your addon with several arguments. The addon can read these arguments using:
    parse arg  ddir, tempfile, action,list,verb ,uri,user, ,
              basedir ,workdir,privset,env_id,transaction,verbose, ,
             servername,host_nickname,home_directory,param1,semqueue,prog_file
    The following describes these arguments (note that several of these arguments are rather arcane or are nearly obsolete):
    ddir
    The data directory, may be host specific (typically, x:\WWW)
    tempfile
    The temporary, "transaction specific" file (i.e.; D:\GOSERVE\TEMP\$341.80) being used by this request
    action
    The action portion of the request (the portion of a request before a ?)
    Example (assuming the GET HTTP method was used by the client)
  • If the request selector is: /sports/getscore?city=boston&date=July+5+1994
  • then the action is sports/getscore
  • list
    • If verb=GET: the partially cleaned up portion of the request selector after the ? (URL decoding is performed).
      In the above example, list would be city=boston&date=July+5+1995
    • If verb=POST: the body of the request -- without URL decoding.
    verb
    Either GET or POST
    uri
    The "universal resource indicator" -- the request-line portion of the original request string, as sent by the client to the server -- without cleanup or encoding (but with "special ! directives" removed). This can be useful if & (and other special characters) may be text returned by GET method client requests -- in which case the partial processing done to the list may cause trouble.
    Note: http GET requests have 3 components: a verb, a request line, and a protocol.
    user
    The numeric IP address of the client
    basedir
    The "base directory", where SRE-http (and probably GoServe) are located.
    workdir
    The fully qualified name of a directory that the addon can use for "temporary" files. (set in the TEMPFILE_DIR variable).
    privset
    The list of user privileges granted to this client.
    env_id
    A substring that can be used when directly accessing the innards of SRE-http (rarely used)
    transaction
    The transaction number
    verbose
    The verbose instruction (1 signifies "say intermediate results").
    servername
    The IP name of the "host" this request is addressed to. This is derived from a HOST: request header, or from the numeric ip address.
    host_nickname
    The SRE-http nickname for the "host".
    home_directory
    The "home directory" (used in ~ replacment)
    param1
    A parameter used for by some older addons (rarely used)
    semqueue
    Semaphore/queue information -- used by the daemon manipulation procedures (as described in DAEMONS.DOC)
    prog_file
    Fully qualified name of the file (containing this addon). If addon was "loaded from macrospace", this will be an empty string.

    5.b.ii) Return Values

    The xxx.CMD procedure should return a simple status message of the form:
    response_code bytes_returned Description Message , file_used
    Where:
  • response_code : the http response code (typically, 200 for GET requests and 201 for POST requests)
  • bytes_returned : the number of bytes transfered (should not include response headers)
  • Description message (optional) : a descriptive message (it's transferred to the post-filter)
  • file_used (optional) : The program that was run (must follow a comma)
  • Examples:
  • 200 12515 Message Succesfully Recieved, D:\GOSERVE\MESSAGES
  • 401 302 Unauthorized access of PRIVATE DATABASE
  • 5.b.iii) Notes

    Return to table of contents


    5.c)Support for CGI-BIN scripts.

    SRE-http supports CGI-BIN scripts (using procedures adapted from Don Meyer's GoHTTP package). To run CGI-BIN scripts, you'll need to: SRE-http's implementation of CGI-BIN has one major improvment: the ability to send pieces of the output as they are created. That is, one can:
    1. wait for the CGI-BIN script to end, and then send all the output it generated
      ... or
    2. send output as it is generated
    The second option is especially useful when large amounts of output are being generated (it avoids some troublesome server-timeout & client-impatience problems). For details see the discussion of the CGI_BIN_PIECES variable in INITFILT.DOC

    SRE-http's implementation of CGI-BIN also includes a few minor modifications:

    Need more information on CGI-BIN ?

    5.c.i) Using CGI-BIN scripts in other subdirectories

    By default, CGI-BIN scripts are assumed to be in the CGI_BIN_DIR directory (say, \GOSERVE\CGI-BIN). Obviously, this can become inconvenient once you acquire a greater-than-small set of scripts. The obvious solution is to place the script files in subdirectories.

    Unfortunately, there does not seem to be a clearly specified method of doing this. With this in mind, SRE-http offers two means of specifying subdirectories containing your CGI-BIN scripts:

    1. Using "aliases".
      This does not require changes to the standard syntax of CGI-BIN request selectors; i.e.;
      CGI-BIN/ASCRIPT/PATHINFO/?args ). The disadvantage is that a seperate alias is need for each script -- even for scripts in the same subdirectory.
    2. Using the portion of the request selector before the CGI-BIN/
      The notion is to include subdirectory information before the CGI-BIN/ (the "this is a CGI-BIN script" indicator). SRE-http will interpret this starting portion as a subdirectory of the CGI_BIN_DIR, or as a virtual directory (in both cases,the usual HTACCESS controls may apply). This permits a powerful division of scripts into directories. The disadvantage is that it doesn't seem to be "spec" (hence may be cumbersome to port).
      Requests for CGI-BIN scripts are often followed by /PATHINFO/. This is typically used by the script (it does not point to the script).

      Example: KIDSTORE/INVENTORY/CGI-BIN/TOYCT?toddler
      This invokes the TOYCT program (with an argument of toddler),
      which is located in (perhaps) \GOSERVE\CGI-BIN\KIDSTORE\INVENTORY\.

    5.c.ii) Notes on creating CGI-BIN scripts

    5.c.iii) Using PERL scripts with SRE-http

    In order to take advantage of a broad spectrum of CGI-BIN scripts, SRE-http can use "non-REXX" processors to interpert CGI-BIN scripts.

    As long as you have the appropriate interepreter installed on your server, it's quite easy to do this. All that you need to do is tell SRE-http (using the INTERPRET_TYPES parameter) which interpeter to associate with different file extensions -- see INITFILT.DOC for a details on how to change the INTERPRET_TYPES parameter.

    Of particular interest is the use of one of the OS/2 PERL interpreters; since there are a slew of PERL scripts out there. In fact, it's not that hard to obtain a PERL interpreter -- see PERL.DOC for details on one we've had some success with.

    Return to table of contents


    6)Miscellaneous SRE-http features

    SRE-http has numerous features that allow you to selectively record and audit requests, upload files, perform "pre-filter" and "post-filter" actions, and sundry trickery and sleight of hand.

    6.a) Special commands, and directives, recognized by SRE-http

    SRE-http offers a set of special commands, and special directives for performing TCP/IP actions, computing statistics on the status of GoServe, and requesting special server actions. These command all start with an exclamation point (!), followed by a word or a phrase.
    The special commands are basically built-in "server side programs". In contrast, the special directives are used to modify SRE-http's logic.

    6.a.i) Currently available special directives

    !force/selector
    The !FORCE special directive instructs SRE-http to suppress the LOGON_FAIL_FILE (and ACCESS_FAIL_FILE) feature when considering the selector. This guarantees that unauthorized clients will be presented with a username/password screen.
    !FORCE is meant to be used in the LOGON_FAIL_FILE and ACCESS_FAIL_FILE, where it provides a means by which a client can try again with a different username/password.

    It is also used to force re-compilation of an HTML document with server side includes (in other words, it suppresses use of the SSI-Cache).

    Hint: To avoid problems with NetScape, use !FORCE/xxx instead of !FORCE?xxx
    Examples:
  • !FORCE/DIR1/INDEX.HTM
  • !FORCE/DIR2/INCS/INDEX.SHT
  • !NOBALANCE/selector
    !NOBALANCE will suppress all attempts at load balancing -- the recieving server will process the request.
    !CGI-BIN_n (n=0,1,2, 3, 4, or 5)
    Overrides the CGI_BIN_PIECES and EMPTY_PATH_OK parameters on a selector specific basis.
    • 0 : Sets EMPTY_PATH_OK=0 and CGI_BIN_PIECES=0
    • 1 : Sets EMPTY_PATH_OK=1 and CGI_BIN_PIECES=0
    • 2 : Sets EMPTY_PATH_OK=0 and CGI_BIN_PIECES=1
    • 3 : Sets EMPTY_PATH_OK=1 and CGI_BIN_PIECES=1
    • 4 : Sets EMPTY_PATH_OK=0 and CGI_BIN_PIECES=2
    • 5 : Sets EMPTY_PATH_OK=1 and CGI_BIN_PIECES=2
    Notes:
  • EMPTY_PATH_OK controls how CGI-BIN PATH_INFO is saved to the OS/2 Environment.
  • CGI_BIN_PIECES enables the the send CGI-BIN (stdout) output as it becomes available mode (either named-pipe mode or FILEREXX mode.)
  • For further details on EMPTY_PATH_OK and CGI_BIN_PIECES, see INITFILT.DOC.
  • Examples:

  • http://foo.bar.net/!CGI_BIN_0/CGI-BIN/TEST-CGI
  • http://bar.foo.gov/!CGI_BIN_2/CGI-BIN/WWWPAGE.EXE?Status

  • !RUN
    The !RUN special directive is used to expedite processing of SRE-http addons. In particular, it is meant to be used with SRE-http addons that have been referenced in one of the KEEP_ADDON. entries.

    Examples:

  • <a href="/!RUN/GIF_TEXT&font=enviro&text=hello">
  • <a href="/!RUN/STATUS?">
  • When !RUN is used, SRE-http assumes the addon has been preloaded into macrospace (hence the KEEP_ADDON. requirement). With this assumption, it is possible to not bother with a fair amount of overhead -- thereby expediting loading and execution.
    For more details, see the description of KEEP_ADDONS. in INITFILT.DOC.
    !norecord?selector
    Instructs SRE-http to NOT record this request. It has the same effect as a NO_POSTFILTER permission. You can disable this option by setting the NO_NO_RECORD parameter.
    !sendas_mimetype_mimesubtype/dir/filename.ext
    The !sendas special action is used to force the server to send the requested file as mimetype/mimesubtype. This can be useful if you wish to create links to unusually named text, or html, files. For example, if you have a series of html documents named CHAP.1, CHAP.2, etc.; you could use links of the form:
    <a href="/!sendas_text_html/mybook/chap.3">Get chapter 3 </a>

    Technical note: !sendas suppresses SRE-http's extension mapping method of determining the mime type (and subtype); and uses the explicitly coded mimetype and mimesubtype instead.
    For information on customizing SRE-http's extension mapping, see the description of MEDIATYP.RXX.

    Special feature If !SENDAS_CHECK_TYPE is used, then if no explicit match can be found, the file will be checked for "all text characters", and if such is the case, will be returned as text/plain. !SENDAS_CHECK_TEXT is simliar, except it checks the file before using the explicit extension (the explicit extension is used if only the file contains non-text characters).

    !delsend/filename.ext
    !delsend is used to send a file from the SRE-http TEMPFILE_DIR "temporary file" directory. More precisely, !delsend will look for filename.ext in the TEMPFILE_DIR, send it (if it exists), and then delete it.

    !delsend is provided to facilitate writing server side programs that generate several output files -- the client would be sent back a response containing !delsend links to these several files.

    Notes

    6.a.ii) Currently available special commands

    !authorize
    SRE-http will send back an "authorization" request. The primary users of this will be server administrators -- since a !AUTHORIZE link will repetitively ask for the client's username/password.
    !ping
    'PINGS' the client.
    !statistics
    View current GoServe statistics.
    !HOST?xxx.yyy.zzz.aaa
    Get information on IP address xxx.yyy.zzz.aaa
    !reset
    Resets various GoServe statistics, and instructs SRE-http to re-read parameter files immediately (only available to OWNERS, or clients with SUPERUSER privileges).
  • If !reset?params, then statistics will not be reset.
  • !save
    Saves various GoServe statistics to the GOAUDIT.80 audit file. This is only available to SUPERUSERs, or to users with a CONTROL privilege.
    !ssi
    Used to access information on the SSI-cache. The following sub-options are recognized:
  • !ssi?status : writes a (fairly technical) status report to the PMPRINTF window.
  • !ssi?status=short : writes a brief status report to the PMPRINTF window
  • !ssi?status=fully_qualified_file_name : writes a report on the SSI-Cache entry (if it exists) for the fully_qualified_file_name
  • !ssi?reset = Resest (clear) the SSI-cache
  • !ssi?remove=fully_qualified_file_name : remove the SSI-Cache entry (if it exists) for the fully_qualified_file_name
  • !ssi?cleanup = Cleans up the SSI-Cache
  • Notes:
  • The client must have SUPERUSER privileges to invoke these !SSI options.
  • After completion of the !SSI request, a short status report (in plain-text format) will be returned to the client.
  • For an illustration of the use of !SSI, see SSICACHE.HTM
  • !variable?var1&var2..
    Returns the value of environment variable var1, etc. (client must have SUPERUSER privileges to use this).
    !write_cache
    saves the record_all_file cache to the record_all_file
    !dir?dirname
    Invokes SRE-http's built in directory lister; which will create a list of all files (and subdirectories) of dirname.
    To modify the various display parameters used by !dir, see the DIR.DOC file, or see the description of the DIR_EXCLUSION, DIR_ACCESS, and DIR_OPTIONS parameters in INITFILT.DOC.

    Notes:

  • The !CREATE option of the AUTO_NAME parameter also uses !dir to generate a directory listing.
  • For a more richly featured directory lister, try the GETAFILE addon for SRE-http.
  • Return to table of contents


    6.b) Common-log, browser, and referer audit files

    In keeping with http server standards, SRE-http will maintain a common-log format audit file. Entries in common-log format audit files have the following structure:
    aclient.somehere.org - username [18/Mar/1996:16:24:08 -0500] "GET /SAMPLES/HELLO.HTM HTTP/1.0" 200 1251

    Besides offering a compact synopsis, a number of tools exist to help you analyze common-log files.

    Furthermore, as an adjunct to the common-log audit file SRE-http can also record the browser used by each request (as well as the referer) in seperate log files.

    Notes

    Return to table of contents


    6.c) Recording ALL requests for files

    You can use the simple-mode configurator to enable the common-log audit file and the RECORD_ALL_FILE running count.
    As an alternative to the common-log audit file, SRE-http maintains a running count of requests in the RECORD_ALL_FILE . This file contains a count of requests, broken down by selector or by filename.
    Record refers to updating a count of the number of times a resource has been requested.
    It does not mean keeping a record of each hit! If you need such detailed accounting, you should use the common-log file (or you can use GoServe's GOAUDIT.80 file).
    For customized auditing needs, you can add your own procedure using the post-filter option (POSTRCRD.80 is an example of such a procedure).

    To enable the RECORD_ALL_FILE, you need to set the RECORD_OPTION parameter. RECORD_OPTION can take the following values:
    NO
    Do not maintain a running count
    YES
    Record requests, but remove the the argument list from the selector (for example; remove ?hi+there in ANIMALS/MONKEY.HTM?hi+there selector
    YES_ALL
    Record requests, do not remove an argument list.
    Caution: Use of YES_ALL can lead to a very large RECORD_ALL_FILE
    FILE
    Record requests by fully qualified file name. In contrast, when YES or YES_ALL are specified, requests are recorded by selector
    Caution: Use of FILE can lead to a very large RECORD_ALL_FILE -- careful use of wildcard entries is recommended.

    RECORD_ALL_FILE: Implementation details

    To speed throughput, SRE-http caches the RECORD_ALL_FILE in memory, with a single thread responsible for updating it (the POSTFCHK thread). This raises several issues:
    1. The cache is written to disk on a regular basis (about once every five minutes). Thus, it is possible for the file version to be somewhat out of date; and it is possible to loose entries (say, if the server crashes).
    2. If you "hand edit" the RECORD_ALL_FILE (say, to add wildcard entries), SRE-http will read it into the cache, overwriting any changes that have been recently made. That is, any requests recieved while you were editing the file will not be recorded.
    3. If the cache grows large (over 1000 entries), SRE-http will start over -- and save the old RECORD_ALL_FILE (you won't loose information, but it will be spread over at a few files). There is one exception: SRE-http will retain "wild card" entries (but will reset their count to 0).

    Given the above: the RECORD_ALL_FILE should be treated as an approximate (albeit probably very accurate) count of requests.
    To avoid loosing a few minutes of entries that will occur when you shut down GoServe, you can issue a !WRITE_CACHE request just beforehand

    Notes

    Return to table of contents


    6.c.i) Suppress recording of repetitive requests

    To reduce the number of false hits from clients re-requesting a URL after a short absence (often caused by the use of a browser's "back" key), a list of current hits can be maintained. If a request selector matches a request in this list of current hits (where the request selector and the client's IP address are both used), the "count of hits" will not be augmented. Specifically, the COUNTER_FILE and the RECORD_ALL_FILE will not be augmented.

    In addition to "re-requests", you may also wish to not record requests from OWNERS (such as the webmaster) -- since these are likely to be due to site maintenance activity, and not due to interest in the content of the requested files.

    INITFILT.DOC contains details on both of these options: see the descriptions of the HIT_CACHE_LEN, HIT_CACHE_DURATION, and HIT_OWNER_SUPPRESS variables (or you can use the simple-mode configurator).

    Notes

    Return to table of contents


    6.d) Load balancing requests across several servers

    SRE-http is shipped with support for a simple form of "load balancing". When load balancing is active, the server will first check the number of active clients. If this exceeds the number specified in the LOADTHRESHOLD variable, then the client is redirected to one of the servers listed in the BackupServerList variable.

    For more complex load balancing schemes (such as may be required in multi-host systems), you can use the BackupServerProc parameter to specify your own load balancer; such as one of the load balancer addons available from the SRE-http home page.

    For details on how to invoke load balancing, see the description of LOADTHRESHOLD in INITFILT.DOC

    Suggestion: If you are interested in "dynamic" load balancing (that uses real-time server-load information); you might be interested in the Balancer dynamic load-balancer.

    Return to table of contents


    6.e) Creating user written pre-filter(s)

    In certain cases, you may wish to have a set of custom written REXX routines examine the request before yielding control to SRE-http. For example, you may wish to shunt some special requests to a variety of different filters.

    To do this, set the PRE_FILTER variable:

    You can specify the name(s) of the pre-filter procedure file(s) by setting the PREFILTER_NAME variable. By default, PREFILTER_NAME="PREFILTR" (which refers to PREFILTR.80).

    When a pre-filter is called, several arguments will be passed. The first three are generated by GoServe, the second to last is generated by SRE-http and is not passed when PRE_FILTER=FIRST, and the last offers a means of transfering results from earlier to later pre-filters.

    parse arg source,request,sel,privs,servername,otherargs
  • source: myaddr port transaction who whoport
  • request: verb selector protocol
  • sel: action '?' awords
  • privset: list of privileges
  • servername -- ip name of server (useful in multi-host environments).
  • Other_args -- results from prior pre-filter.
  • The pre-filter should return a comma delimited list:
    status , status message
    Where status is either 0 or 1:
  • 0= No response to client, SRE-http should continue processing.
  • 1=Response to client, SRE-http should exit (perhaps with response recording)
  • The Status message will be stored as PREFILTER_RESULT. Note that the REPLACE PREFILTER_RESULT keyphrase can be used to display this status message.

    Notes

    Return to table of contents


    6.e.i) Using a pre-filter to invoke the GoRemote package

    The PREFILTR.80 file that comes with SRE-http contains a pre-filter that enables support for the GoRemote package. GoRemote (the "server remote control" package bundled with GoServe) is typically called as a special filter. By using this pre-filter, GoRemote will automatically be called (instead of SRE-http) when a GoRemote specific action is requested.

    To do this, you'll need to:

    1. Make sure you have the GOREMOTE package (GOREMOTE.ZIP), and unarchive it to either the root of your data directory, or to the GOREMOTE/ subdirectory of your data directory. You'll need GOS32.GIF, GORM*.HTM and GOREMOTE.* files. Since there are about a dozen GORM* files, placing them in a subdirectory is asthetically preferable.
    2. Copy GOREMOTE.80 to your GoServe working directory. If GOREMOTE was installed in a GOREMOTE/ directory, copy GOREMOTE.HTM, GOS32.GIF and GORMOK.HTM to the root of your data directory
    3. Make sure that PREFILTR.80 (the one that comes with SRE-http) is also in your GoServe working directory.
    4. Using CONFIG2.HTM (or by direct modification of the INITFILT.80 file), set the PRE_FILTER variable to:
      • YES = If you want to limit access to GoRemote:
      • Clients with SUPERUSER or INHOUSE privileges will be able to view filter settings and statistics.
      • Clients with SUPERUSER privileges will be able to change settings.
      • FIRST = If you want all clients to have access to GoRemote

      You should also set PREFILTER_NAME='PREFILTR.80'
    5. Set up a link to GOREMOTE (or GOREMOTE/GOREMOTE if you installed GOREMOTE in a GOREMOTE subdirectory).
    6. You do not need to make any other modifications, either to SRE-http, or to the GoRemote files!
    Notes:

    Return to table of contents


    6.f) Creating a user written Post-Filter

    In certain cases, you may wish to execute some additional actions after responding to a client's request. For example, you may wish to record client specific information in a custom designed audit file. Or, you may wish to e-mail responsible parties when certain events occur (such as a file upload). Or you may wish to do both!

    To accomodate such needs, SRE-http can be told to call a series of "post-filters". As with the "pre-filter", these post-filters are custom designed REXX procedures.

    To enable post-filter calling, set the POST_FILTER variable:

    You can specify the names of the (possibly several) post-filter procedure files by setting the POSTFILTER_NAME variable. By default, POSTFILTER_NAME="POSTFILT".
    To specify several post-filters, just enter a space delimted list. For example POSTFILTER_NAME="POSTMAIL.80 POSTRECRD.80 " -- they will be executed in the order of appearance.
    Note that if there is no extension, and no ending period, an extension of .80 is assumed (or .nnn if you are using a non-80 serverport).

    When a post-filter is called, several arguments will be passed:

    For a simple example of a post-filter, see the POSTFILT.80 file that comes with SRE-http. For more useful examples, see POSTMAIL.80 (an e-mail alert routine) and POSTRCRD.80 (a client info recorder).

    6.f.i) Using SREF_MAILIT to issue "event specific" alerts

    The POSTMAIL.80 that comes with SRE-http uses the SREF_MAILIT procedure (contained in the SREFPRC1 macrospace procedure library) to e-mail "event specific" alerts through an SMTP Gateway. A different (set) of individuals can be notified for each of a set of events, where events are identified by the request selector (with wildcard matching supported), or by the requesting client's IP address. POSTMAIL.80 contains detailed instructions on how to generate these e-mail alerts.

    Notes:

    Return to table of contents


    6.g) Uploading and deleting files

    SRE-http provides two built-in "actions" for uploading files, one HTTP method for uploading files, and one HTTP method for deleting files. Each of these has it's own advantages and disadvantages:
    1. GET_URL: A built-in action for uploading files from another HTTP server
      • Requires that the client has ready access to an HTTP server.
      • Given this access, uploading is quick and easy.
    2. PUT_FILE: A built-in action for uploading files from the client's own machine
      • Requires that the client be using a browser that supports the type=file FORM attribute (such as NetScape 2.01 and above).
    3. The PUT method.
      • Requires that the browser (or other software) know how to issue a PUT method HTTP request. The DOPUT.CMD program illustrates the use of the PUT method.
    4. The DELETE method. This can be used to delete files from your server. As with the PUT method, it requires a browser that knows how to issue DELETE method HTTP requests.
    General Usage notes:
    1. GET_URL.

      Set up a URL with the GET_URL action, followed by the URL of the file you want to get, and the "local name". The request selector should look like: GET_URL?url=http://a_url&file=relative_file_name

      For example:

      • GET_URL?url=http://www.pc.com/price.lst&file=pcprice.1
      will copy price.lst (on the server at www.pc.com ) to a file named PCPRICE.1 (in the upload directory).
    2. PUT_FILE.

      Create an HTML FORM with a type=file attribute. For example:

      <FORM enctype="multipart/form-data" ACTION="/put_file" METHOD="post">
        Send this file <INPUT TYPE="file"  name="relative_filename">
        <INPUT TYPE="submit" VALUE="send file">
      </FORM>
      
    3. The PUT and DELETE HTTP methods are, by default, not permitted. To allow PUT or DELETE methods, you must include a PUT or a DELETE permission in the permission list of the appropriate entry (for the requested selector ) in the ACCESS_FILE.
    4. As with the more commonly used GET and POST methods, selectors in PUT and DELETE method requests are relative to the data directory (or to a local virtual directory).
    5. You can allow PUT to overwrite preexisting files by setting PUT_NO_OVERWRITE=0 (in SREFILTR.80). If PUT_NO_OVERWRITE=1, then if the selector maps to a pre-existing file, an error message will be sent back to the client.
    6. PUT and DELETE will not be permitted if ACCEPT_RANGES='NO' (ACCEPT_RANGES is set in INITFILT.80).

    PUT_FILE and GET_URL usage Notes

    Return to table of contents


    6.h) Automatic generation of response headers

    If desired, SRE-http will parse a requested document, extract all LINK and META HTTP-EQUIV elements found in the <HEAD>, and create the appropriate response headers. This is especially useful in response to HEAD requests.

    To do this, set the value of the AUTO_HEADER variable:

    Notes

    Return to table of contents


    6.i) Defining new media types

    SRE-http recognizes a list of the most frequent MIME media types (using the extension of the requested file). If you want to add to this list, you can do so by editing the MEDIATYP.RXX file located in your GoServe directory.

    Additions to MEDIATYP.RXX may be new media types (say, an experimental MIME type), or they may be assignations of a non-standard extension to a common MIME type (say, .DOCUMENT to text/plain).

    Notes

    Return to table of contents


    7) Some useful facilities provided with SRE-http

    SRE-http comes packaged with several useful tools; including a text-search engine, a speciallized file-transfer'er, a message box utility, and client customization tools.

    7.a) Searching files using DOSEARCH

    The DOSEARCH addon for SRE-http is a generic search utility written in REXX. It searches one (or several) ASCII text files for "paragraphs" that satisfy a set of conditions that are specified in a "search string", and returns an HTML document containing these paragraphs.

    DOSEARCH looks for "search strings" within "paragraphs", and uses "meta commands" and "target specific" instructions to structure the search.

    By default, a paragraph is defined as being all text between blank lines. Alternatively, one can define paragraphs as single lines, or as delimited by any arbitrary character sequence.

    A search string is comprised of "targets" There are two kinds of targets: subwords and phrases.
    Each space delimited entry in the search string is treated as a seperate subword, except..
    for phrases, which are delimited by parenthesis; for example: (xx yy zz)
    .... and phrases must be matched precisely.

    7.a.i) Search algorithims.

    DOSEARCH has two modes: SIMPLE and LOGICAL.
    Simple mode with highlighting.
     Two meta-commands and 4 "target specific"  instructions are recognized.
            Meta-commands are signified by  *&  or *\ at the beginning
            of the search string.
                 *&   means "find paragraphs that match ALL targets in
                    the search string"
                 *\  means "find paragraphs that match NONE of the targets in
                     the search string"
         If there are no meta-commands, the following target specific
         commands are recognized.
              &   means "paragraphs MUST have this target"
              |   means "accept paragraph if it has this target"
                  Note that | is the default (assumed if no target specific
                  command entered).
              \ means "paragraph must NOT have this target"
              % means "accept paragraph if it does NOT have this target"
    
        Summarizing: to be a "found" paragraph:
          Test 1a) Any (of several) | targets must be present,    or
               1b) All of the % targets must be missing
             If pass test 1a and 1b, then
               2a) None of the \ can be present, and
               2b) All of the & must be present
         If present, all & and | targets will be highlighted.
         Note that if there are no % targets specified, test 1b is ignored.
    
    Logical expression mode without highlighting
    The user enters REXX-like logical expression using the following operators:
  • & : AND operator
  • | : OR operator
  • \ : NOT operator
  • @ : XOR operator
  • ( ) : to group expressions.
  • A sequence of words without any operators is treated as a phrase -- to treat each word as a seperate subword, put ( ) around each one. Basically, when using this mode, be liberal in your use of ( ).

    7.a.ii) DOSEARCH Options

    DOSEARCH supports a number of options. These options are easily set in DOSEARH.HTM, or by modifying the ALIASE(es) that invoke DOSEARCH.

    Technically speaking, the options appear in an option list, with each option seperated by an & character.....

    The option list should have the structure:
    option_name=option_value&option_Nam2=option_value2&...

    The DOSEARCH options are:

           DELIM : The paragraph delimiter.
                       " "  or 0 = blank lines   (the default)
                       "$"      = Each line is a paragraph
                       other     = User specified delimiter
           LINE  : Maximum number of lines to display of each "found" paragraph.
                     If 0, no lines displayed (a summary will be displayed)
                     Default is to display all lines.
           NUM   : YES=Display the line or paragraph number,
                   NO=Don't (default=YES)
           BAR   : YES= Seperate each paragraph/line by a horizontal bar,
                   NO=Don't (default=YES)
           EXPERT: YES= Use "logical expression mode",
                   NO=Use simple mode (Default=NO)
           FILE  :  FILE=filename
                    A file to search (either relative to the data directory,
                    or in a "local" virtual directory).  You can include as many
                    FILE options as desired (each entry will be searched in turn).
                    Furthermore, * and ? wildcards can be used.
           SEARCH: The search string
           CASE  : If YES, then search is case sensitive (default is NO)
    

    Examples

    Simple mode examples
    ( The best car)
    Finds paragraphs containing the phrase the best car
    car truck motorcycle
    Finds paragraphs that contain one (or more) of car, truck or motorcycle
    dog cat & store (pet pig)
    Finds paragraphs that contain one (or more) of dog, cat or the phrase pet pig; and that also contain store
    *& computer price memory
    Finds paragraphs that contain computer, price, and memory (must have all of them, but can be in any order)
    Logical mode examples
    The best car
    Finds paragraphs containing the phrase the best car
    car | truck | motorcycle
    Finds paragraphs that contain one (or more) of car, truck or motorcycle
    ( (dog & cat) | (pet pig) ) & stores
    Finds paragraphs that contain stores, and that contain either both dog and cat, or the phrase pet pig

    7.a.iii) Using DOSEARCH.HTM

    DOSEARCH.HTM is an HTML document that directly calls DOSEARCH -- it does not require using an ALIAS or an <ISINDEX> element. It also provides an easy means for setting a number of DOSEARCH options. If you are familiar with HTML FORMS, you can customize (and rename) DOSEARCH.HTM. In particular, to facilitate searches (say, of a specific document) many of the TYPE fields can be set to be HIDDEN.


    Return to table of contents


    7.b)The SENDFILE facility

    SENDFILE is an SRE-http facility that transfers files to a client. It can record the number of times this file has been completely sent.

    SENDFILE is invoked with a request selector of:
    SENDFILE?selector&forcetext=1&COUNTER=option_list
    (the &forcetext=1 and &COUNTER=option_list are optional)

    SENDFILE will ...

    1. Check for the existence of a file corresponding to the selector (the file that the selector maps to is assumed to be relative to the data directory or relative to a "local" virtual directory).
    2. Look in the SENDFILE_FILE for an occurence of the selector. If none exists, create an entry.
    3. If the selector exists (and the entire file was sent), the "count of requests" for this selector is augmented. That is, if the client killed the connection half way through, the count will not be augmented.
    4. If a COUNTER=... options is present, COUNTER.RXX will be used (with the option_list).
    5. If SAVE_STATE=1 the number of bytes sent and the size of the file are stored in a SENDFILE_STATUS variable in the "request specific" cache.
    6. A successful transfer will have the "number of bytes sent" equal to the "size of the file".
    7. SAVE_STATE is set in SREFILTR.80 -- see INITFILT.DOC for details.
    Notes

    Return to table of contents

    7.c) Using Message Boxes

    SRE Filter's built-in message box facility provides a mechanism for storing messages from clients. SRE-http provides a few simple built-in tools for manipulating these messages boxes. These include:
    1. A message writer. The message writer is invoked with a request selector of:
      MESSAGE?messbox=messagebox&field1=value&..&fieldn=value
    2. Two simple facilities (VIEWMESS and ASKMESSBOX) are provided for viewing an entire message box.
    3. Message box access can be controlled through the use of message box privileges.
    Of perhaps greater interest, the FORUM, READMAIL, and MAILIST SRE-http addons make extensive use of these SRE-http messages boxes.

    7.c.i) The Message Writer

    The message writer is usually invoked with an HTML FORM with ACTION="/MESSAGE" (see SAMPMBOX.HTM for an example). In addition to an INPUT element identifying the "message box", you can specify a number of fields (the field name, and it's value, are written to the message box using an e-mail type format). Several of these fields are treated specially, as explained below:
    MESSBOX (recommended)
    The particular message-box to put the note into. If a MESSBOX field is not included, the note will be put into the ANYONE message-box. If MESSBOX names a non-existent message-box, a new message-box will be create.
    NAME and SUBJECT (recommended)
    These are expected, if not included a dummy value will be created. Note that each message is automatically given a TIME and DATE stamp.
    FAX, E-MAIL, ADDRESS, and PHONE (optional).
    The name, and value of these fields are written to the message box. Note that the E-MAIL field is explicitly used by SRE-http's FORUM addon.
    PASSWORD and DURATION
    The PASSWORD and DURATION fields are used by the FORUM addon to control "delete privileges" for a given message. PASSWORD will be encrypted before being written to the message box file. Note that messages boxes are not designed for security (the encryption is weak), so don't use them when confidentiality and integrity are important.
    INDEX
    Including an INDEX=YES field will cause MESSAGE to update an index file (with an .IDX extension). These .IDX files are used by the FORUM addon to speed up response. Note that this .IDX file creation will occur after responding to the client (it's a post-filter type of action) -- the client will not notice any adverse impact from inclusion of an INDEX=YES field (but other clients may be impacted by the extra workload!)
    BROADCAST_LIST
    If included, SRE-http will simultaneously e-mail (through the SMTP gateway you set with the SMTP_GATEWAY variable) copies of your message to a list of e-mail addresses. This list, a file with name given by BROADCAST_LIST, must be in (or under)the MESSBOX_DIR. Furthermore, the client must have SUPERUSER or BROADCAST privileges, or a privilege equal to the value of BROADCAST_LIST. If not, SRE-http will send authorization request back to the client.
  • Example: if BROADCAST_LIST=CLASS10.LST, then the client must have a CLASS10.LST client privilege.

  • Note that the BROADCAST_LIST file (i.e.; CLASS10.LST) should contain valid e-mail addresses, one per line.
    BROADCAST_ONLY=YES
    Do not write to message box, just BROADCAST
    BROADCAST_NAME=My_name
    Used in the "from" field of the e-mail message.
    WRITE_TO_FILE=YES
    If included, SRE-http will write an "E-mail" like file (containing From, To, Date, and Subject fields; followed by the Contents) to your MAILBOX_DIR directory. These files are designed to be read, and processed as an "e-mail message to a list-server", by the MAILLIST (and READMAIL) SRE-http addons.
    If you are broadcasting to many recipients, the use of WRITE_TO_FILE in conjunction with the MAILLIST addon, rather then the BROADCAST_LIST, is recommended.
    .
    Other names (optional).
    The name, and value, of these fields will be written to the message box.
    The difference between "other names" and the PHONE, ADDRESS, and FAX "optional" fields is merely in the order of appearance.
    RESPONSE_FILE
    The RESPONSE_FILE field is used to indicate a file contaning a "response" to a posted message. If no RESPONSE_FILE is included, a very simple generic response will be returned.
    MESSAGE (recommended)..
    The (multi-line) message. It's written last of all.

    7.c.ii) Viewing Message boxes with VIEWMESS and ASKMESSBOX

    The direct means of reading a message box is to simply view it using the OS/2 TYPE command (message boxes are ASCII files, with a .LOG extension, located in the MESSBOX_DIR directory). SRE-http also provides a simple mechanism for remotely viewing these message boxes.
    A request selector of VIEWMESS?messbox=message_box will cause SRE-http to transfer the message-box-file to the client.
    SRE-http also provides a simple routine that generates an HTML document that displays a list of all available message-boxes, and let's the client choose which one to view. This routine can be invoked with a request selector of
    askmessbox?An+optional+header
    The "optional header" is displayed as an <H2> at the top of the document.

    7.c.iii) MESSBOX Privileges

    To access a message box with MESSAGE (or with the FORUM addon), the client must have "message box specific" privileges. These are specifed by including (in the client's privilege list), entries of the form:
    MESSBOX=a_message_box.
    For example: MESSBOX=FORUM1 would give (read/write) access to the FORUM1 message box. Note that this is in addition to any "selector and FILE specific" access controls you may have imposed.

    The a_message_box portion of the privilege can contain the * "wildcard" character. For example, MESSBOX=FORUM* would provide access to FORUM1, FORUM2, etc.

    On installation, SRE-http includes MESSBOX=* in the PUBLIC_PRIVS parameter (the effect of which is to allow open access all message boxes). You'll need to remove this from PUBLIC_PRIVS if you want to limit access to message boxes!

    Notes


    Return to table of contents


    7.d) The SRE-http procedure library

    SRE-http uses a procedure library (that is loaded into macrospace) to provide many functions. Several of these routines may be quite useful to the ambitious REXX programmer. These procedures include: See SREFPRC1.DOC. for a complete description of these procedures.


    Return to table of contents


    7.e) Client-specific and browser-specific customization

    SRE-http provides two addons that facilitate the creation of customized documents. Both of them are invoked with an <!-- INTERPRET FILE --> server side include:
  • BYCLIENT.RXX: ByCLIENT.RXX is used to specify a set of files to be #INCLUDEd on a client-specific basis. Identification of clients is based on either a special cookie, the client privileges, or IP address.
  • ByBROWSR: ByBROWSR.RXX is used to specify a set of files to include on a browser-specific basis. Browsers are identified by their user-agent; which are then matched (possibly using * wildcards) to one of several file-mapping rules.
  • Suggestion:For a highly dynamic (cookie based) means of customizing documents, check out the CUSTOMIZ addon.


    Return to table of contents


    8) SRE-http addons

    In addition to all the features/options/utilities packaged with SRE-http, there are an ever-increasing number of SRE-http addons available.
    Note that the SRE-http addons use a somewhat non-standard interface. However, given the flexibility and power of the GoServe/SRE-http API , these addons are often considerably faster then equivalent CGI-BIN scripts, and can often provide services (such as server push) that would be almost impossible with standard CGI-BIN
    You can find & download the latest SRE-http addons from the SRE-http home page.

    The following (non-exhaustive) list of currently available addons is provided to tease your appetite: Did we say non-exhaustive? Some of the other addons include:
  • GoSWISH: an advanced front-end to SWISH (complements SRCHINDX),
  • Customiz: a client-customization utility (using cookies)
  • GIF_TEXT: a text-to-gif converter (using numerous fancy-multi-colored fonts)
  • WWWthumb: a create & deliver grahpical thumbnails utility
  • and much more!

  • Return to table of contents


    Appendix 1) User-configurable SRE-http files.

    SRE-http uses several user-configurable files. These files contain logon information, access and accounting information, and instructions toassist in conducting server side includes. In most cases, SRE-http's simple mode configurator (CONFIG0.HTM) or intermediate mode configurator (CONFIG2.HTM) can be used to modify these files. In some cases, modification of these files requires the user to edit them with a text editor.

    This section describes these files. For more detailed and up-to-date descriptions, please see the sample parameter files shipped with SRE-http (i.e.; the VIRTUAL.IN, ALIASES.IN, and ALL_FILE.CTL in the DATA\ subdirectory).


    A1.a) Initialization Parameters File (INITFILT.80)

    SRE-http contains a number of parameters that can, and should, be changed to fit the needs of the user. To do this, you'll need to change the the INITFILT.80 (or INITFILT.nnn, if you are using a port nnn).

    Changes to the INITFILT.80 can be accomplished in several ways:

    1. You can edit it with your favorite text editor. You might want to examine INITFILT.DOC for a detailed discussion of the meaning, and possible values, of each variable included in INITFILT.80.
    2. You can use SRE-http's simple or intermediate mode configurator. To run either of these configurators, point your browser at http://your.server/configur.htm and follow the instructions. The simple mode configurator allows you to change the most important parameters, while the intermediate mode let's you change almost all the parameters.
      If you need to modify the few parameters that the intermediate mode (purposefully) ignores, you'll have to hand edit the INITFILT_FILE (and read INITFILT.DOC first!).
      You must be an OWNER, or have SUPERUSER privileges, to access these configurators. Extremely cautious administrators may wish remove CONFIG2.CMD and SREFCONF.CMD, as well as all the *CFG.CMD files, from the SRE-http ADDON_DIR directory (typically, \GOSERVE\ADDON.

    Return to table of contents


    A1.b) Replacement strings file (REPSTRGS_FILE)

    The replacement strings file contains "custom" replacement strings that can be used by the REPLACE keyphrase.

    Each line of this file (except lines beginning with a semi-colon, which are treated as comments) has two fields:

    Example

    CONTACT Contact me soon!
  • CONTACT is the variable_name , it may contain a .HOST_IDENTIFIER.
  • Contact me soon! is the replacement string
  • When a <!-- REPLACE CONTACT -- > keyphrase is encountered, it is replaced by Contact me soon!

  • Note on multi-line replacements
    The REPLACE keyphrase can be used for multi-line replacements (note that for replacements of more then a few lines it is probably wiser to use the INCLUDE keyphrase).

    To specify a multi line replacement, just enter multiple occurences of the same label -- the values will be appended. Example:

  • MYADDRESS <address> My address is:</address>
  • MYADDRESS 1800 M St NW <br>
  • MYADDRESS Washington, DC <br>
  • would generate 3 lines (when a <!-- REPLACE MYADDRESS --> keyphrase is encountered).
    Providing arguments to custom replacement strings
    You can provide arguments in the REPLACE KEYPHRASE that will be substituted into the replacement string. These arguments should follow the varname, with each URL encoded word being an argument. For example:
    To be substituted, the replacment string should contain %n substrings, where n is an integer. For example: Note that a maximum of 9 arguments should be supplied. All unused %n substrings will be removed from the replacment string. Thus, if a %6 is included in the replacment string, and only 5 arguments are provided, the %6 is removed.
    Notes

    Return to table of contents


    A1.c) The HITS and COUNTS counter file (COUNTER_FILE)

    The counter file is used to record the number of requests (also known as the number of hits) for this document. It is used whenever a REPLACE HITS or a REPLACE COUNTS keyphrase is processed.

    The counter file has syntax:
    HOST_NICKNAME//AURL CTS MESSAGE_1 , MESSAGE_2

  • HOST_NICKNAME is optional.
  • AURL is a selector corresponding to an HTML document
  • CTS is the current count of the number of requests.
  • Message1 and message2 are used by HITS to display the # of requests.
  • When a REPLACE HITS keyphrase is encountered, SRE-http looks in the counter file for an aurl that matches the request selector If it finds a match, the CTS, MESSAGE_1 and MESSAGE_2 values are extracted. CTS is then augmented by one and written back to counter file.

    Lastly, a "count of hits" string is included in the the requested document. This string will have the form:
    Message_1 cts+1 Message_2
    a generic message_1 is used if message_1 and message_2 are missing

    For REPLACE COUNTS, the same procedure is followed, except message_1 and message_2 are ignored.

    If the request selector does not match any of the aurl's, a new entry will be created (with CTS=1 and message_1 and message_2 left blank).

    Examples

    INDEX.HTM  66 You are visitor # , since <!--replace creation_date  -->}
    PROJECTS/PROJ1.HTM  19
    CALC/CALC.HTM  9
    OVERVIEW.HTM  13
    BRIDGES//VERMONT/COVERED.HTM  15
    
    Note the use of the message_1 and message_2 blocks in the entry for INDEX.HTM. Also note the use of a REPLACE keyphrase in the message_2 block.

    Notes

    Return to table of contents


    A1.d) The RECORD_ALL_FILE counter file

    The RECORD_ALL_FILE (with default name RECRDALL.CNT) is used to record all actions, such as transfers of files and requests for server side processing. It is used when RECORD_OPTION is enabled.

    The structure of the RECORD_ALL_FILE is similar to the COUNTER_FILE, with a few exceptions:

  • All requests are tracked (including requests for HTML documents, requests for .GIF files, and Server Side Processing requests).
  • Instead of a message_1 , message_2 following the aurl cts , the date of the last successful request for this selector written.
  • RECORD_ALL_FILE may contain "wildcard matches" -- typically subdirectory entries.
  • Example

    PROJECT/PROJ1.HTM  9 03/01/96
    TEMP/ZFVL5615.LST  1 02/28/96
    DOSEARCH  4  2/29/96
    OVERVIEW.HTM 7 03/01/96
    TEMP/*  3 03/01/96
    ZOO// SCHEDULE.HTM 120 05/03/96
    
    Note the TEMP/* entry is used as an wildcard match; if RECORD_OPTION="YES" or "YES_ALL", then all requests for files in the "temporary directory" will be recorded in the TEMP/* entry (the assumption is that one is typically not interested in specific information on temporary files). Also note TEMP/ZFVL5615.LST -- it was written when a file in the "temporary" directory was requested, and before the TEMP/* entry was added to the RECORD_ALL_FILE (the server administator must have added the TEMP/* entry on 2/29/96!).

    Also, note that the date is the date of the most recent "allowed" request.

    Notes

    Return to table of contents


    A1.e) The SENDFILE_FILE counter file

    The SENDFILE_FILE counter file (with default name SENDFILE.CNT) is used by the SENDFILE facility to record transferals.

    The structure of the SENDFILE_FILE is:
    URL_NAME #_of_times_sent date_of_last_transfer

    Example

    CTEMP/FOO.HTM  4 07/06/96
    GOS/GO1.HTM  6 07/06/96
    EAA.HTM  20 07/06/96
    /SAMPLES/SRCHINDX.DOC  2 07/07/96
    /SAMPLES/TESTTABL.HTM  1   07/06/96
    ZOO//  TIGERS.DOC 5  06/07/96
    
    Notes:


    A1.f) The ACCESS_FILE access control file

    The ACCESS_FILE (with default name ALL_FILE.CTL) contains access information on a SEL-specific basis. Each entry in the ACCESS_FILE can contain several types of information:
    1. A selector . This may contain the wildcard (*) character (required)
      It also may be preceded by the host nickname.
    2. A space delimited resource privileges list (optional).
    3. A space delimited permissions list (optional)
    4. A SEL-speciflc realm name (optional)
    5. A SEL-specific access failure file (optional)
    6. An advanced options file (optional)

    The structure of each entry in the ACCESS_FILE is:
    host_nickname// SEL resource_privilege_list , permissions_list , realm_name , failure_file, adv_opts_file

    Special Entries: REALM-specific, additional resource privileges

    Each realm can have associated with it a list of realm-specific, additional resource privileges. If available, this (space delimited) list is added to the selector specifid resource_privilege_list.
    The syntax of these special entries is:
    !REALM realm_name realm_privilege_list
    Note that if there is no !REALM entry for a given SEL-specific realm, then just the resource_privileges list is used.
    Also note that an empty resource_privileges list implies "all clients are allowed access", but only if there is no realm-specific, additional resource privileges for this SEL.

    Examples

    CLIENT1/PROJ1.DOC   VENUS ,  NO_SSI , PLANET_2
    BATMAN//  CLIENT1/*   ROBIN GOHTAM , NO_CODE , REALM_BAT , D:\GOSERVE\NOBATS.HTM
    WORKAREA/MAPLE.IN   INHOUSE MERCURY
    WORKAREA/SYCAMORE.IN MERCURY VENUS MARS &EARTH
    USAGE.TXT   NO
    WEATHER*  INHOUSE ,  , CLIMATE_GROUP
    READ.ME *
    VIEWMESS?messbox=SECRET2*  INHOUSE
    PUBLIC/*   *
    GUEST/*  *  ,  NO_SSI NO_SSP
    UPLOADS/* * , PUT DELETE
    OREGON/*  ,  , REALM1
    imgs/*.gif * , cache no_virtual no_alias no_postfilter no_htaccess
    !REALM REALM1 BIGFOOT SASQUATCH
    ZOO// EMPLOYEES.HTM  MANAGER
    
    ACCESS_FILE summary

    A1.g) The virtual directory file

    The VIRTUAL_FILE (with default name VIRTUAL.IN) is used to specify local or remote virtual directories.

    Each line of the VIRTUAL_FILE contains:
    HOST_NICKNAME SEL_fragment TARGET limitation_list , user:pwd .

    1. The optional host_nickname dicates which host the entry applies to.
    2. If the limitation_list is not enabled, or if the request is one of the types listed in the limitation_list , then the SEL_fragment is compared to the beginning of the request selector.

      If it matches, the TARGET directory, along with the portion of the request selector that follows this SEL_fragment is used as the root directory for this request.


    Note that there are two kinds of virtual directories: local and remote.
    LOCAL
    Local directories are located on, or are LAN accessible from, the server.
    Local directories should be specified using a fully qualified directory name.
    REMOTE
    Remote directories are "web" accessible from the server. That is, requests to other http servers can be used to obtain files in these directories (assuming the other http server allows unrestricted access to the desired URLS).
    Remote directories should be specified using a complete URL, that must start with http://.
    Remote directories are only used for document requests.
    Examples:
    LOCAL/DISK1/PROJECTS d:\work\PROJECTS*
    A selector of /LOCAL/DISK1/PROJECTS/BOB/PLANE.HTM is "mapped" to D:\WORK\PROJECTS\BOB\PLANE.HTM
    REMOTE1/ http://www.their.org/
    If a selector of REMOTE1/TURNIPS.HTM is recived, then SRE-http will attempt to obtain REMOTE1/TURNIPS.HTM from the http server at www.their.org. This retrieval does not require a "redirection" by the client -- all the work is done by SRE-http!
    PHYSICS// STRINGS/ d:\theory\basic\strings !HTML
    Request to a host with a host nickname of PHYSICS, that begin with STRINGS/, are mapped to files in d:\theory\basic\strings. However, only document requests will use this entry -- cgi-bin, SRE-http addon, and file upload requests will not use this entry.
    ALASKA/ r:\states\alaska
    A request for ALASKA/BEARS.HTM "maps" to R:\STATES\ALASKA\BEARS.HTM .
    If R: is a LAN drive, and the server is not logged on, this could cause trouble!

    Notes


    A1.h) The usernames and passwords file (USER_FILE)

    Each entry in the the USER_FILE (with default name USERS.IN) contains an optional host nickname, a username, password, and privilege information. It is used in two cases:
    1. When logon is required of a client.
    2. When information on privileges is needed.

    The structure of the USER_FILE is:
    host_nickname NAME PASSWORD PRIV1 PRIV2 ...
    Note that the PRIV1 is required, while other privileges (PRIV2, etc.) are optional. Also note that the host_nickname is optional

    Examples
      DANIEL A11S34WZ SUPERUSER
      MIKE IOWAMAN  CONSULTANT
      JILL GOBOTS      INHOUSE
      JOHN  KIOWA    INHOUSE  VENUS
      FLOWERS//  CHARLES BROWN PRICELIST
     ANONYMOUS *  PUBLIC
    
    In the above example:

    Return to table of contents


    A1.i) ALIASES file (ALIAS_FILE)

    The list of ALIASES is contained in the ALIAS_FILE (with default name ALIASES.IN)

    The ALIAS_FILE contains lines having the structure:

     host_nickname alias_target replace_string
    where host_nickname and alias_target should have no embedded spaces.
    Note that the host_nickname is optional.

    To support several special features, several special character strings may be placed in either the alias_target or replace_string:

    Examples

    ; This first example replaces a  "bad request selector  " with
    ;a correct HTML file names
    
    HOME INDEX.HTM
    
    ; The next example would be used if CHECKCAT.HTM was a gateway to a
    ; "searchable index" (CAT.DAT being the file that is actually searched).
    
    CHECKCAT.HTM?* DOSEARCH?FILE=CAT.DAT&SEARCH=*
    
    ; This redirects all request selectors   that begin with YAH to yahoo
    YAH* http://www.yahoo.com/
    
    ; This converts request selectors  only to a host with a host_nickname of ANIMALS,
    ANIMALS//  CATS/*  FELINES/DOMESTIC/*
    
    ; This sends some "not in data directory" documentation to the client.
    WHATGOT !TRANSFER=E:\PRODUCTS\DESCRIBE.DOC
    
    ; This performs a "wildcard match with substitution, in combination with
    ; a redirection.  Typically, this would be called as WEBSEARCH?find_this
    WEBSEARCH?* http://www.webcrawler.com/cgi-bin/WebQuery/?*
    
    ; Identify all .HTM files in MANUALS/ as variant resources, and use 
    ; MANUALS/MANUALS.LST as the variant list (MANUALS.LST must contain a
    ; PATTERN: MANUALS/* entry)
    MANUALS/*.HTM !NEGOTIATE MANUALS/MANUALS.LST
    
    

    CAUTION


    Unless you explicitly want to "alias" selectors that otherwise would map to actual files,
    we recommend that all "wildcard match aliases" have "targets" that do NOT map to pre-existing files.
    The point is that a "wildcard matching" alias match could convert a legitimate selector (that points to a real file in the data directory or a virtual directory) into something else -- rendering it impossible to obtain this "real file".

    Return to table of contents


    Appendix 2) Summary

    AP.a) Known limitations of SRE-http

    The currently known limitations of the SRE-http package include: In other words, SRE-http is designed as relatively full-featured package for non-cutting edge small-to-medium load sites being maintained by non-professionals

    A2.b) Summary of SRE-http Features

    The following summarizes most of SRE-http's features:

    Return to table of contents


    Appendix 3) Terminology

    The following terms are used in this documentation. Please forgive any non-standard uses you may encounter (better yet, let me know).

    You might also want to review the basic terminology for extended descriptions of a few key concepts.

    The \ and / characters
    The / character is used in URLs to "indicate hierarchical structures, such as directories". It is equivalent to the \ OS/2 (and DOS) character. Thus, / that appear in a request selector are converted into \ when SRE-http needs to find a file.
    Action
    The action is the "location" portion of a request selector.
    • If the GET method was used -- it's the part after the first / (that ends the domain name) and before the first ? (or the end of the request selector , if no ? is present).
      Examples:
      • In http://rpbcam.econ.ag.gov/calc/calc.htm,
        the action woud be calc/calc.htm.
      • In: http://foo.bar.net/workdir/myfile.htm?not+readyl,
        the action would be is workdir/myfile.htm.
      • In:http://young.kids.com/toys/demo?dogs+cats,
        the action would be toys/demo
    • If the POST method was used, the request selector (not including a leading / character) is the action (there should never be a ? in a POST method request string). All other information is contained in the "body" of the request. SRE-http assumes that all POST method requests are for Server Side Processing.
    Addon directory
    The addon directory (with a value given by the ADDON_DIR parameter) is where SRE-http's external procedures should be stored. For example, several external procedues shipped with SRE-http (i.e; the simple and intermediate configurator, and the DOSEARCH text search engine) should be located in this directory.
    Alias -- converting selectors
    SRE-http uses "aliases" to perform textual substitution. Typically, an alias is used to replace a simple request selector with a more complicated one. In addition, aliases are used for several special purposes: implementing "searchable indices", document redirections, and specification of the location of CGI-BIN scripts.
    Alias -- virtual hosts
    A given IP address can be used by several hosts: including the host associated with the "canonical" name, and hosts associated with one of several aliases. Unfortunately, TCP/IP requests arrive with only the numeric IP address, making it difficult for the server to determine to which host the request is directed to. However, HTTP 1.1 compliant browsers (such as NetScape 2.01) will include a HOST: request header containing the host to whom this request is directed; and SRE-http can use this request header (if present) to determine the proper host.
    Client
    The Client (sometimes referred to as the requester) is the individual requesting a URL -- where the URL may point to a file, or may invoke Server Side Processing. Typically, the request was generated by someone running a Web Browser, and clicking on a link or submitting a FORM.
    CGI-BIN
    CGI (Common Gateway Interfact) - BIN is a standard by which server side ("gateway") programs can be invoked. It uses environment variables to pass data, as well as standard input and output. SRE-http supports CGI-BIN, using procedures adapted from the GoHTTP package. One nice feature of SRE-http is an ability to call alternate (non-REXX) interpreters for specific scripts (such as PERL when a script has a .PL extension).
    Although this support should permit use of many CGI-BIN scripts, the use of the "native" SRE-http interface (using REXX procesures that are passed information in the argument list) offers advantages to ambitous programmers
    The Default Data Directory
    The Default Data Directory is the root directory of your web site -- it (and it's subdirectories) are the usual locations for the HTML documents, images. In simple cases, the GoServe data directory is used as the the Default Data Directory. On multiple-hosts server, each host can be granted it's own Default Data Directory.

    A simple example might help:

    • a request selector of MYFILES/PROJECT1/BIGTASK.HTM is recieved
    • your default data directory is E:\WWW
    then, SRE-http will look in E:\WWW\MYFILES\PROJECT1 for the BIGTASK.HTM file.

    A Note On Data Security in SRE-http

    In general, the only files available for clients are those in (or under) the Data Directory. Exceptions to this rule include:
  • Files explicitly transferred by user-supplied executable code (i.e; REXX procedures). Note that user-written Rexx code may be invoked by a CGI-BIN script or a SRE-http addon, or by INTERPRET, #EXEC and SELECT keyphrases.
  • A !TRANSFER "alias" can be used to access files from anywhere on the server.
  • Files specified by literal PUBLIC_URLS.
  • Virtual directories can point to anywhere.
  • ELEMENT
    ELEMENT's are the constituent parts of an HTML document. They dictate the structure, layout, and display characteristics of an HTML document. ELEMENTS are identified using HTML tags, where an HTML tag consists of text within < and > characters. Often, a start and stop tag are used.

    For example:

    • <H1> Welcome </H1> : defines a "level 1 Header"
    • <A href="/morejunk/stuff.htm"> More info? </A> : defines a link to a local document.
    File Names: Fully qualified and Relatve
    SRE-http can treat File Names as "fully qualified" names or as "relative" names. Fully qualified names should be fully pathed -- containing the complete path to the file (sometimes the drive letter will be missing, in which case the GoServe drive is usually assumed). Relative names are assumed to be in subdirectories of some assumed default. For example, when a client asks for a file name, it is assumed to be relative to the data directory (or to a matching virtual directory).
    FORMS
    The HTML FORM element is used to create a fill-in form. A FORM element typically contains an ACTION attribute, as well as a number of elements that provide editable fields (for the client to fill in or to otherwise select). Typically a form is designed in tandem with a server-side program.
    Although not the only way to generate a Server Side Processing Request (one can use sets of detailed URLS), FORMS are probably the easiest for the average client to deal with.
    Hosts
    It is possible for one server machine to handle requests to multiple IP addresses. This "multiple hosting" (also refered to as multi-homing) can occur either by having the server handle several numeric IP addresses, or by the existence of multiple aliases for a single numeric IP address.
    Host nickname
    SRE-http uses host nicknames when identifying which host a parameter applies to. Besides being a convenience, it allows several hosts to point to identical resources (given that each host has the same host nickname).
    HTTP : Hyper Text Transfer Protocol
    The rules by which web-aware clients and servers communicate.
    HTTP method (also known as the VERB)
    SRE-http recognizes five HTTP methods: GET, POST, HEAD, DELETE, and PUT.
  • GET requests contain all information (action name and optional parameter list) in the request string (with the parameter list placed after a ? character)
  • POST requests contain only the action name in the request string (the "body" of the request contains the parameter list). Currently, SRE-http assumes that all POST requests are requests for Server Side Processing.
  • HEAD method requests are sent "meta information" on the requested file, the file itself is not sent.
  • PUT method requests are used to upload information to the file pointed to by a request selector . SRE-http will only honor PUT method requests if specific permissions are granted for that selector (as set in the ACCESS_FILE).
  • DELETE method requests are used to delete (the file pointed) a requested selector . SRE-http will only honor DELETE method requests if specific permissions are granted for that selector (as set in the ACCESS_FILE).
  • HTML
    Hyper Text Markup Language (HTML) is the protocol used to create documents designed for display by WWW browsers (such as Netscape, Mosaic, MS Internet Explorer,and IBM Web Explorer). Basically, HTML consists of a set of formatting commands, which are included in a file containing the text to be displayed.
    Note that to be recognized by SRE-http as an HTML file, a file must have an .HTML, .HTM, .SHT, or a .SHTML extension (though it's easy to add extensions to this list).
    Although there is a formal HTML standard (currently 2.0, with 3.2 under discussion), it is not rigidly adhered to. In particular, NetScape has it's own extensions; many of which are not recognized by other browsers. For example, IBM's Web Explorer ver 1.03 recognizes the CENTER element, and the SIZE attribute of the HR element, but not the FRAMES element.
    Imagemap
    An image map (sometimes called a mappable image) is an inline image which, when clicked with the mouse, returns a pixel location to the server. Typically, the server then translates the pixel location into a URL, and redirects the client to that URL (which might be on an all together different server). SRE-http supports image maps using the MAPIMAGE/ "indicator string", along with NCSA and CERN style "MAP" files to use in translating pixel locations to URL's.
    Note that some browsers (such as NETSCAPE 2.0) support "client side mappable images", which require no action by the server (but which require an HTML element not understood by many other browsers).
    IN-HOUSE
    IN-HOUSE refers to clients that are "in-house"; specifically, they have had INHOUSE privileges assigned to them. The idea is that one often wants to differentiate members of your "organization" from the rest of the world, and place fewer limits on what they can obtain from the server.
    Macrospace
    Marospace is best thought of as a global procedure cache used by OS/2. When REXX programs are asked to load external procedures, they first look in macrospace. For efficiency reasons, SRE-http makes extensive use of macrospace.
    Privileges
    SRE-http uses "privileges" to control access to selected resources. There are two classes of privileges:
    1. "resource" privileges which are assigned to server resources (such as files, or directories), using entries in the ACCESS_FILE.
    2. client privileges which are granted to clients (based on their IP address, or their explicit username/password).
    Resource privileges can be ONE_OF privileges ("or" privileges") or MUST_HAVE privileges ("and" privileges). Client privileges may be static (assigned using INITFILT.80 or the USER_FILE), or may be dynamic (set on the fly as a function of the client's earlier requests).
    Request selector
    When a client sends a request to a server, it contains three components: the http method, the selector, and the http protocol.
    For example: GET /FOO1/BAR.HTM HTTP/1.0
    In this documentation, the term request selector refers to this middle component, with leading / stripped, and character decoding performed.
    In the above example, the request selector would be FOO1/BAR.HTM.
    Note that if a GET HTTP method was used, the request selector may contain information following a ? (for example, SHOWINFO?database=Prices&item=apples).
    Response Header
    The response header contains "meta-information" about the requested document. GoServe automatically sends a few response headers (such as content length and file creation date). In addition, using the LINK and HTTP-EQUIV tags in the <HEAD> portion of your document, you can instruct SRE-http to automatically add other response headers.
    Searchable Index
    A searchable index is a document that points to a file that is to be searched. One can think of a searchable index as:
    .. the front end to a gateway program, and the file you search as the database represented by the document.
    Typically, a searchable index contains the <ISINDEX> element.
    SEL-specific
    SEL-specific means "a piece of information specific to a request selector . Clarifying, this means that SRE-http will use the value of the request selector to find information (such as resource privilege information). Thus, SEL-specific resource privileges really means "privileges associated with a particular request selector (coming up with terminology isn't easy ....)

    Note: the SRE-http documentation sometimes uses the term URL-specific to mean SEL-specific (it's a holdover from an earlier revision of the documentation).

    Server Side Includes
    Server side includes refer to on-the-fly modifications of requested HTML documents. Inclusions can be static, in which case the use of server side includes is mostly to benefit the creation and maintenance of these documents. Dynamic includes refer to information that changes, such as the current time and date -- obviously, these can not be specified in the document a priori, but must be determined at the time of request.
    Server Side Processing
    Often, the server provides computational services to clients. An example of this is a client specified search of a database, with the results displayed using an 'attractive" HTML document (generated just for this search). Or, the client might request a map of some portion of the U.S, displaying a choice of geographic features. Such actions require the server to execute some kind of program (perhaps a REXX program, or perhaps some other executable).
    SRE-http addons, and CGI-BIN scripts, are typical means by which Server Side Processing is provided to clients.
    Server Side Processing Request
    A Server Side Processing request is, generically speaking, "a set of parameters, sent by a client, and an action name, that dictate Server Side Processing". Often these are generated by a client responding to a FORM element in an HTML document (alternatively, they may be "hand coded" into a URL included in a HTML document). Server Side Processing requests are recognized by:
  • If the GET method was used: the presence of a ?options string after a "non-HTML" action name; where options contains the "set of parameters".
    Notes:
    • SRE-http assumes that, regardless of the presence of a ? in the request selector , all action names that end with a .HTM, .HTML, .SHT, or .SHTML are requests for HTML documents (that is, are not requests for server side processing),
    • Actions which require no parameters must have a ? trailing the action name.
    • Server side programs are sometimes referred to as gateway programs.
  • If the request selector starts with CGI-BIN/ , then it's assumed to be a request for a CGI-BIN script.
  • If the POST method was used, SRE-http assumes a Server Side Processing Request, with the request selector treated as the action (which identifies the program to run), and the "set of parameters" contained in the "body" of the request.
  • STEM variables.
    STEM variables are REXX's way of implementing arrays. A STEM variable consists of a NAME.INDEX1.INDEX2 syntax (that is, with a . used to indicate "array indices"). SRE-http uses STEM variables for several of it's configuration parameters.
    SUPERUSER status
    Clients with SUPERUSER status are granted full privileges. Currently, this includes:
  • Never having to logon (SUPERUSERs are INHOUSE users)
  • Automatically granted CONTROL privileges (can perform remote server resets)
  • File uploads are never blocked.
  • URL: Uniform Resource Locator.
    URL's are a scheme for specifying Internet resources using a single line of printable ASCII characters. A URL should contain a protocol, domain name, port number (optional), the location of the resource, and an (optional) option list (following a ?).
    In this documentation, the URL term connotes "a link coded into an HTML document".
    Upon selecting such a link, the client sends back a request string that contains the verb (the HTTP method), the request selector , and the http protocol (typically, HTTP/1.0)
    Example:given a URL of http://www.econ.ag.gov/calc/calc.htm
    the resulting request selector would be calc/calc.htm
    URL character encoding
    Since several characters have very specific meanings in the HTTP protocol (such as the & and the > characters), and since spaces are not allowed in request strings, a set of character encoding rules are needed to transmit such characters.
    Caution: When handcoding URL's with an option list following a ? and seperated by & characters, you must encode the & character.
    For example:
    • INCORRECT: <A HREF="/GETAFILE?dir=PUBLIC/&showdate=YES">
    • CORRECT: <A HREF="/GETAFILE?dir=PUBLIC/&amp;showdate=YES">
    Web Browser
    A Web Browser is a program that can communicate over the Internet, and that is capable of displaying HTML documents. NetScape, Mosaic. and IBM Web Explorer are examples of Web Browsers.
    Wildcard matching
    Wildcard matching use the * as a wildcard character in a natural fashion.
    Examples:
      /JOE/*  will match /JOE/FOO.HTM
      /JOAN/SRCH.HTM?*  will match /JOAN/SRCH.HTM?search+me
      /JOAN/SRCH.HTM?*  will NOT match /JOAN/SRCH.HTM
             (/JOAN/SRCH.HTM* will match BOTH these examples)
      /PETS/*INDEX.HTM will match
         /PETS/INDEX.HTM, /PETS/CAT/INDEX.HTM and /PETS/PUPPY/LAB/INDEX.HTM
                 but will NOT match
         /PETS/CAT/PUREBRED.HTM
    

    SRE-http files that support wildcard matching are: ALIAS_FILE, RECORD_ALL_FILE, SENDFILE_FILE, ACCESS_FILE, and the PUBLIC_URL stem variables.
    The COUNTER_FILE does not support wildcard matching.

    Wildcard matching with substitution
    Wildcard matching with substitution is similar to wildcard matching. However, rather then being a "many to one" match, it is a "many to many" match.
    The basic rule is that both a "target" and a "replacement" should contain * characters. When a wildcard match between the target and a candidate occurs, the "covered portion of the candidate" is inserted into the "replacement". In other words, the * character in the replacement is deleted, and the "covered portion" is inserted.

    Example:

    • If the request selector is: project/jillwork.htm
    • and there is an alias of: project/* /research/ongoing/*
    • SRE-http will convert the request selector to /research/ongoing/jillwork.htm
    GoServe Working Directory
    The GoServe Working Directory in the directory containing GOSERVE.EXE (typically, \GOSERV). You should copy the various SRE-http files to it (see the installation section for details).

    Return to table of contents


    For questions, contact Daniel Hellerstein.
    Summary of SRE-http Features<