Hypermail 2.0
Contents:
Hypermail is a program that takes a file of mail messages in UNIX mailbox format and generates a set of cross-referenced HTML documents. Each file that is created represents a separate message in the mail archive and contains links to other articles, so that the entire archive can be browsed in a number of ways by following links. Archives generated by Hypermail can be incrementally updated, and Hypermail is set by default to only update archives when changes are detected.
Each HTML file that is generated for a message contains (where applicable):
- the subject of the article,
- the name and email address of the sender,
- the date the article was sent,
- links to the next and previous messages in the archive,
- a link to the message the article is in reply to, and
- a link to the message next in the current thread.
In addition, Hypermail will convert references in each message to email addresses and URLs to hyperlinks so they can be selected. Email addresses can be converted to mailto: URLs or links to a CGI mail program.
To complement each set of HTML messages, four index files are created which sort the articles by date received, thread, subject, and author. Each entry in these index files are links to the individual articles and provide a bird's-eye view of every archived message.
Hypermail was originally developed and designed by Tom Gruber for Enterprise Integration Technologies (EIT) in Common Lisp. It was later rewritten in C by Kevin Hughes while at EIT. Hypermail is now being maintained by Kent Landfield <kent@landfield.com>.
To see what Hypermail can do, take a look at these Hypermail-produced archives:
Usage: hypermail [options]
Options:
-a URL : URL to other archives
-b URL : URL to archive information
-c file : Configuration file to read in
-d dir : The directory to save HTML files in
-i : Read messages from standard input
-l label : What to name the output archive
-m mbox : Mail archive to read in
-M : Use metadata
-n listaddr : The submission address of the list
-o keyword=val: Set config item
-p : Show progress
-s htmlsuffix : HTML file suffix (.html, .htm, ..)
-t : Use Tables
-T : Use index tables
-u : Update archive by one article
-v : Show configuration variables only
-V : Show version information and exit
-x : Overwrite previous messages
-1 : Read only one mail from input
-L lang : Specify language to use (de en es se fi )
Using the flags -h, or -? with Hypermail will display this usage summary.
Input and Output Options:
-i,
-m "mailbox",
-d "directory",
-c "file"
To tell Hypermail what mailbox to read in, use the -m option. If articles will be sent to Hypermail through standard input, use the -i option. Note that the -m and -i options can't be used together! By default, Hypermail will look for a file called mbox to read its articles in from.
The -d option specifies the directory to put the HTML files and index files that are created into. If the directory doesn't exist, a new one will be created with the name that is specified. If the -d option isn't used, Hypermail will look for a directory with the same name as the mailbox or will create one if needed.
example 1: hypermail -m "wu-ftpd" -d "/wu-ftpd"
example 2: cat "/var/spool/mail/wu-ftpd" | hypermail -i
- This example reads the articles in wu-ftpd and will save the output in the /wu-ftpd directory.
- This reads the file /var/spool/mail/wu-ftpd from standard input and will save the output in a directory called wu-ftpd in the same directory Hypermail was run from.
Note that Hypermail can only read messages in the UNIX mailbox format! Such archives are typically RFC 822 mail messages appended to each other that look similar to this:
From john@foo.com Mon Jan 1 00:01:30 1994
Date: Mon, 1 Jan 1994 00:01:15 PDT
From: john@foo.com
To: everyone@foo.com
Subject: Hello, world!
Hi, everyone, just saying hello!
From someone.else@foo.com Mon Jan 1 00:02:00 1994
Date: Mon, 1 Jan 1994 00:01:45 PDT
...
The messages are typically separated by lines in this format:
From wu-ftpd@wugate.wustl.edu Fri Jul 1 00:18:20 1994
The -c option tells Hypermail to read in settings from a configuration file. By default, the program will attempt to read settings from a file called .hmrc in the user's home directory if it exists.
In the configuration file, variables are set in the following manner:
variable = number
variable = "string"
The complete set of variables that Hypermail recognizes is described in the Configuration Options section.
Archive Interface Options:
-l "label",
-b "About URL",
-a "Other Archives URL"
The -l option tells Hypermail what to call the archive - the name that is specified will be in the title of the index pages so users know what sort of messages are being archived.
The -a option includes a link labelled "Other mail archives" in the index pages to any specified URL. This way users who are looking at the archive have the opportunity to go to pointers to other mail archives. By default, this will be a pointer to the parent directory in which the archive files reside.
The -b option includes a link labelled "About this archive" in the index pages to any specified URL. This way users who are looking at the archive have the opportunity to go to information about the archive.
example: hypermail -l "WU-FTPD Development Archives"
-a "http://www.landfield.com/wu-ftpd/"
-b "http://www.landfield.com/wu-ftpd/mail-archive/"
In the index files for the archive, the above setting will produce something like this:
(top of page)
WU-FTPD Archives
(list of indexed articles below)
Updating Options:
-x,
-u
The -x option tells Hypermail to explicitly overwrite any previous HTML files that may exist. Use this option only when it is desirable to completely rewrite the entire archive.
The -u option tells Hypermail to update the archives by one message only. With this option, only one email message will be read in from a file or standard input. This message will be added to the end of the existing HTML file archive and will be integrated into it by links and cross-references. All archive index files will be regenerated to include the new message.
Important! - If the -u option is used and Hypermail is reading from a mailbox file, Hypermail will assume that the file contains only one article, no matter how many files the mailbox may actually contain. Make sure that Hypermail is given only one article when using this option.
If the mailbox that is being read from is an archive that new messages are always being added to, don't use either of these options. Hypermail will then read in all the messages given it but will only write new messages that have been appended to the mailbox.
example 1: cat "one.letter" | hypermail -i -u -d "/wu-ftpd/mail-archives"
example 2: hypermail -i -u -m "one.letter" -d "/wu-ftpd/mail-archives"
example 3: hypermail -m "mailbox" -d "/wu-ftpd/mail-archives" -x
example 4: hypermail -m "mailbox" -d "/wu-ftpd/mail-archives"
- This tells Hypermail to take the article it receives from standard input
and integrate it with the archive under the wu-ftpd/mail-archives
directory. If no archive exists, a new one will be created with the specified
letter as the first file of the archive.
- This does the same thing, except that the letter is read in from a file that contains only that letter.
- With these options, Hypermail will read in the articles from
mailbox and write over any existing files in the
wu-ftpd/mail-archives directory if they exist. If no archive
exists, a new one will be created.
- With these options, Hypermail will read in the articles from
mailbox and only write new articles - it will not overwrite
any existing archive files.
Note that no matter what options are specified, the index files are always
rewritten. The date when Hypermail was last run is included in index pages,
so it's easy to tell when the archive was last updated.
Miscellaneous Options
-p
-v
-V
The -p option shows a progress report as Hypermail reads in and writes out messages - the number of files that Hypermail is reading and writing and the file names of the directory and files created are shown. This information is written to standard output.
The -v option shows the configuration variables and their values that Hypermail would use if it was run with the same configuration file and command line options. This is useful when starting up a new list or modifying a list configuration file. Once the information is displayed, Hypermail terminates and not actual processing occurs.
The -V option prints the Hypermail version information. Once the information is displayed, Hypermail terminates and not actual processing occurs.
Hypermail has many variables that can be set as environment variables or as variables in the specified configuration file. For instance, using the C shell, one could define variables in this manner:
setenv HM_MBOX /home/john/my_mailbox
setenv HM_FILEMODE 0600
In the configuration file, variables must be in lowercase and separated by their values with an equals (=) sign. Blank lines and lines beginning with the # character are skipped:
mbox = "/home/john/my_mailbox"
filemode = 0600
While the example uses quotes ("), they is not required when used in the configuration file.
Below is a list of configuration variables that can be defined. Boolean numbers can have the value of 0 or 1.
- HM_CONFIGFILE "filename"
- This is the default configuration file to read settings in from. This
can only be specified as an environment variable. If the first character
is "~", Hypermail will look for the file under the current user's home
directory.
- HM_MBOX "filename"
- This is the default mailbox to read messages in from. Define this with
a value of "NONE" to read from standard input as the
default.
- HM_LABEL "label name"
- Define this as the default label to put in archives.
- HM_HTMLSUFFIX "suffix"
- Use this to specify the html file suffix (html, htm, shtml, ..) to
be used when Hypermail generates the html files. This is dependent
on local needs. Do not put a '.' in the value. It would result in
"file..html", probably not what you want.
- HM_LANGUAGE "language-id"
- This is a two-letter string specifying the default
language to use. Set this the value of the language
table you wish to use when running and generating
archives.
Current supported languages:
de - German
en - English
es - Spanish
fi - Finnish
fr - French
is - Icelandic
se - Swedish
- HM_USELOCK [ 0 | 1 ]
- Controls whether to use hypermail's built-in locking mechanism.
By default, this option is set to 1. Set it to
0 if you have an external locking mechanism,
like, for example, when using procmail or smartlist.
- HM_LOCKTIME "number-of-seconds"
- The number of seconds that a lock should be honored when processing
inbound messages before it is overridden.
- HM_USEMETA [ 0 | 1 ]
- This option allows you to use metadata to store the content type
of a MIME attachments and, later on, when a user browses the
attachment, send back this information in the HTTP Content-Type
header. When set to 1, the Content-Type header of a
MIME attachment will be stored in a metadata file. Let us say that
the MIME attachments for a message are stored in directory
att-num. The metadata for those attachments will
then be stored in directory att-num/.meta. If a
MIME attachment is stored in file att-file, its
metadata will be stored in file att-file.meta. This
convention is directly compatible with the Apache server handling of
metadata.
- HM_ARCHIVES "URL"
- This will create a link in the archived index pages to the specified
URL. Define as "NONE" to omit such a link.
- HM_ABOUT "URL"
- This will create a link in the archived index pages to the specified URL. Define as "NONE" to omit such a link.
- HM_DIR "directory"
- This is the default directory that Hypermail will look for when creating and updating archives. If defined as "NONE", the directory name will be the same name as the mailbox read in.
- HM_DEFAULTINDEX "type"
- This is the index that is intended to be the one that users see first when entering the archive. Possible values are "date", "thread", "subject", and "author".
- HM_REVERSE boolean_number
- Defining this variable as 1 will reverse-sort the article entries in the date and thread index files by the date they were received. That is, the most recent messages will appear at the top of the index rather than the other way around.
- HM_USETABLE boolean_number
- Defining this will cause Hypermail to generate an index menu in HTML table format at the top and bottom of each page.
- HM_INDEXTABLE boolean_number
- Setting this variable to 1 will tell Hypermail to generate
a message index Subject/Author/Date listings using a table
format. Set to 0 if you want the standard Hypermail index
page look and feel.
- HM_PROGRESS [ 0 | 1 | 2 ]
- Define as 1 or 2 to always show a progress report as Hypermail works. Defined as 2 shows more information about the
attachment files created.
- HM_ATTACHMENTLINK "attachment-link-format"
Format of the attachment links.
%p - full path to the attachment
%f - file name part only
%d - directory name only
%n - message number
%c - content type string
- HM_SHOW_MSG_LINKS boolean_number
- Define as 1 to put the individual message links at the top of the individual message pages. Define as 0 to produce pages without the Next, Previous, Reply, In-reply-to, etc. links.
- HM_SHOWHEADERS boolean_number
- Define this as 1 to show the article header lines in the archived HTML files. These lines typically include the "To:", "From:", and "Subject:" information found in most email messages.
- HM_SHOWREPLIES boolean_number
- Define as 1 to show all replies to a message as links in article files.
- HM_SHOWHTML boolean_number
- Define as 1 to show the articles in a proportionally-spaced font rather than a fixed-width (monospace) font.
- HM_SHOWBR boolean_number
- Define as 1 to place <br> tags at the end of article lines. Otherwise, all non-quoted article lines will word wrap. This
only takes effect if HM_SHOWHTML is defined.
- HM_IQUOTES boolean_number
- Define as 1 to italicize quoted lines.
- HM_SHOWHR boolean_number
- Define as 1 to place horizontal rules before and after articles.
- HM_OVERWRITE boolean_number
- Define as 1 to make Hypermail overwrite existing archives by default.
- HM_READONE boolean_number
- Set this to 1 to specify there is only one message in the input.
- HM_INCREMENT boolean_number
- Define as 1 to read one article only and append it to existing archives by default.
- HM_DISCARD_DUP_MSGIDS boolean_number
- Set this to 0 to accept messages with a Message-ID matching that
of a message already in this archive. By default such messages
are discarded.
- HM_REQUIRE_MSGIDS boolean_number
- Set this to 0 to accept messages without a Message-ID header.
Set this to 1 to discard messages without a Message-ID header.
By default such messages are discarded.
- HM_DATEFORMAT strftime-date-format
- Format used in strftime(3) call for displaying dates.
See strftime(3) for the valid conversion specifications.
- HM_THRDLEVELS number
- This specifies the number of thread levels to outline in the thread index. For instance, if HM_THRDLEVELS is 2, replies to messages will be indented once in the index, but replies to replies, etc., will only be indented once as well.
- HM_EURODATE boolean_number
- Define as 1 to display article received dates with days before months instead of months before days.
- HM_DIRMODE octal_number
- This is an octal number representing the permissions that new directories are set to when they are created. If the archives will be made publically available, it's a good idea to define this as 0755. If files will be updated incrementally with sendmail, you should make sure the owner of the archives is a sendmail trusted user. See sendmail documentation for more information.
- HM_FILEMODE octal_number
- This is an octal number representing the file permissions that new files are set to when they are created. If the archives will be made publically available, it's a good idea to define this as 0644. If files will be updated incrementally with sendmail, you should make sure the owner of the archives is a sendmail trusted user. See sendmail documentation for more information.
- HM_MAILCOMMAND "command"
- This specifies the mail command to use when converting email addresses to links. Variables can be used in constructing the command string:
- $TO represents the address to send mail to,
- $SUBJECT represents the subject that is being replied to, and
- $ID represents the message ID of the article that is being replied to.
If defined as "NONE", email addresses will not be converted to links in articles. A possible command one could use is "mailto:$TO",
but this could easily be changed to specify a CGI program such as "/cgi-bin/mail?to=$TO". A CGI mail program is included with the source which can be used for this purpose.
- HM_MAILTO address
- The address of the contact point that is put in the HTML header line
- <LINK REV=made HREF=mailto:MAILTO>
The <LINK...> header can be disabled by default by setting HM_MAILTO to "NONE".
- HM_DOMAINADDR "domainname"
- Set this to the domainname you want added to a mail address appearing
in the RFC822 field which lack a hostname. When the list resides on the
same host as the user sending the message, it is often not required of
the MTA to domain-ize these addresses for delivery. In such cases,
Hypermail will add the DOMAINADDR to the email address. If defined as
NONE, this feature is turned off.
- HM_IHTMLHEADERFILE "path"
- Define path as the path to a file containing valid HTML formatting statements that you wish to included at the top of every index page. Hypermail will print this file as the header of the index so make sure it contains <HTML>, <HEAD>, and <BODY> and other statements that suit your local customized needs.
- HM_IHTMLFOOTERFILE "path"
- Define path as the path to a file containing valid HTML formatting statements that you wish to included at the bottom of every index page. Hypermail will print this file as the trailer of the index so make sure it contains at a minimum a </BODY> and </HTML> statement.
- HM_MHTMLHEADERFILE "path"
- Define path as the path to a file containing valid HTML formatting statements that you wish to included at the top of every message page. Hypermail will print this file as the header of the message so make sure it contains <HTML>, <HEAD>, and <BODY> and other statements that suit your local customized needs.
- HM_MHTMLFOOTERFILE "path"
- Define path as the path to a file containing valid HTML formatting statements that you wish to included at the bottom of every message page. Hypermail will print this file as the trailer of the message so make sure it contains at a minimum a </BODY> and </HTML> statement.
- HM_HMAIL "list submission address"
- This is the email address used to send a new message to a hypermail archive. "NONE" means don't use it. Since this is different for each hypermail archive, you should probably leave it set to "NONE" here, and let it be specified at runtime by command-line parameters in the list specific configfile.
- HM_BODY "HTML <BODY> statement"
- This is the <BODY> line to use when generating the HTML pages.
Define as "NONE" to use the builtin <BODY> line by default.
- HM_SHOW_HEADERS "list of RFC 822 Headers to display"
- Define the list of headers to be displayed if the variable HM_SHOWHEADERS
is set to 1 (ON). If it contains the special character ``*'' hypermail will
display all header lines. This is a comma or space separated all on a single
line such as
show_headers = From,Subject,Date,Message-ID
or they can be listed individually or any combination of.
show_headers = From
show_headers = Subject
show_headers = Date
show_headers = Message-ID
- HM_INLINE_TYPES "indicate which data types should be inlined"
- This is the list of MIME types that you want inlined as opposed to
simply linked into the message. They can be listed individually on
multiple lines or comma or space separated on a single line.
inline_types = image/gif image/jpeg
or
inline_types = image/gif
inline_types = image/jpeg
- HM_IGNORE_TYPES "indicate which types of attachments to ignore"
- This is the list of MIME attachment types that you do not want to
do anything with. They are quietly ignored. They can be listed
individually on multiple lines or comma or space separated on a
single line.
ignore_types = text/x-vcard application/x-msdownload
or
ignore_types = text/x-vcard
ignore_types = application/x-msdownload
- HM_TEXT_TYPES "list of types to be the same as text/plain"
- This is a list of MIME types that you want hypermail to treat
exactly as if they were text/plain. They can be listed individually
on multiple lines or comma or space separated on a single line.
text_types = text, text/plain, message/rfc822
- HM_PREFERED_TYPES "multipart/mixed types to present"
- When mails using multipart/mixed types are scanned, this list of
MIME types defines which part you want presented in the result.
prefered_types = text/plain, text/html
Settings are processed in this order:
- From the program's hard-wired internal defaults (specified in options.h),
- From runtime environment variables,
- From command-line options,
- From the configuration file.
It is important to note that this is different from many programs. In Hypermail's
case what you put on the command line DOES NOT override what is
specified in the configuration file. This may change in the future.
Filenames: In the specified directory, articles will be read out in the order that they were read in from a mailbox or standard input. Filenames start at zero and increase in this fashion: 0000.html, 0001.html, 0002.html, etc. In the same directory:
- date.html is the index of articles sorted by the date they were received by the system's mail daemon.
- thread.html is the index of articles sorted by thread first, then the date they were received.
- subject.html is the index of articles sorted by subject. Any "Re:" prefixes in front of subjects will have been stripped out.
- author.html is the index of articles sorted by the first word of the author's name. If the author's name can't be determined, their email address will be substituted.
- One of the above files will be called index.html and is the default index that users can go to when entering the archive.
Sorting: In the date and thread index files, note that these lists are sorted by the date the articles were received by the system's mail daemon, not by the date they were written on. The order of articles in the date index may not necessarily match the order in which the article files are written and linked together. Because of this, it is a good idea to make sure the mailbox is sorted by date with the most recent messages towards the bottom.
Running Hypermail automatically: All that's needed to start archiving email messages is to set up Hypermail to do incremental updates in your /etc/aliases file. Here's what an entry might look like (the last line is one unbroken line):
#
# WU-FTPD Mailing List Archives
#
wulist: "|/usr/local/bin/hypermail -i -u -d /wu-ftpd/mail-archive -l \"WU-FTPD Mailing List Archive\""
After adding the entry, make sure newaliases is run to update the
mail aliases. This entry will run Hypermail and update/create the archive whenever
a new message is received. Hypermail also works well as a cron job.
Because sendmail may run Hypermail as different users, you will
want to make sure that archive directories and files are made readable and writeable
by a trusted sendmail user (or read/writable by everyone if you can't do that)
when they are created. This will ensure that there will be no problems incrementally
updating the archive.
Including HTML in messages: One can include formatted HTML in message bodies by enclosing the HTML with the <HTML> tag (in either uppercase or lowercase). This tag must be on a line by itself:
This text will not be parsed...
<html>
this text will be parsed as HTML.
</html>
This text will not be parsed...
There is no limit to how often the <HTML> tag can be used in an article.
If you are are looking for more information on Hypermail and its features and
developmental status, checkout The
Hypermail Development Center. Additional documentation and the hypermail
development list archives are available there.
Hypermail is available free of charge under GNU Public License. More details
about the GPL are available at http://www.fsf.org/copyleft/gpl.html.
The source is available via FTP from
ftp.landfield.com/hypermail/hypermail.tar.gz. This is a link to the tar file of the most current version.
The Hypermail Development Center also has beta development versions of hypermail available from time to time.
I would like to thank Tom Gruber,
who originally designed and developed Hypermail in Common Lisp, for the basis
of a GREAT tool.
I'd also like to thank Kevin Hughes for
developing the initial C version of Hypermail. Kevin also provided a great
deal of assistance with restarting the current hypermail development.
There are a great deal of people that also contributed to Hypermail's development.
The Hypermail Development Center Credits page is an attempt to let you know just who they are.
Hypermail development is currently being fostered by
<Kent Landfield>.
hypermail(1),
hmrc(4),
Hypermail List Configuration File.
and
Customizing Hypermail Pages
Please send any feature requests, bug fixes, and comments on Hypermail
to <hypermail-bugs@landfield.com>.
Last updated October 9, 1999