The FreeBSD Diary

Providing practical examples since 1998

If you buy from Amazon USA, please support us by using this link.

Creating a digest and archive for a majordomo mailing list 6 November 1999

Need more help on this topic? Click here
This article has no comments
Show me similar articles

This article describes how I added a digest and an archive to a majordomo mailing list. It will show you how to do this for a new list and how to add the features to an existing list. I also have instructions on how to install and

I've been using DSL for almost two years now. I was part of a trial group which were the first to get this great technology. As the trial went commercial, I created a mailing list for the people involved. The list is open to everyone and includes both users and people at the phone company and various ISPs. The list is good in that it's independent of any supplier and we have direct lines to people within the providers. Unfortunately, when I set up the list I did not start an archive and a digest.

The list has grown considerably from when it started in April 1999 and now in November, the time has come to provide an archive and digest. I first created a new test list to establish the steps needed to utilize an archive and a digest. Then I used this knowledge to reverse engineer the changes into an existing list.

What's an archive?

An archive contains messages which have been sent to the list. Users can request these files using the majordomo get command. To see more information on this command, send a message to majordomo at your domain with "help" in the body of the message.

What's a digest?

You are probably familiar with the function of a mailing list. A person sends a message to the list and a copy of that message is forwarded to each person on the list. If ten messages are sent to the list, a subscriber will receive ten messages. However, you can create a digest version of the list. If you subscribe to the digest, you will receive a single message containing all ten message. Actually, the digest defaults to sending out one message a day. That message contains all the messages sent that day. There are also size considerations, but all of these options are within the digest list configuration file.

To get a list of the digest and archives available for a particular list, send a message to majordomo with "index <listname>" in the body of the message, where <listname> is the name of the list in question.

The examples

Now I'm going to divide this topic into two parts. The first example will show how I added a brand new list and then implemented the digest and archive. The second example will be the addition of a digest and archive to an existing list.

A new list

This example deals with the creation of a new list (newdumm) which uses both an archive and a digest.

The first step was to follow the the example for creating a new list in my original majordomo article. Remember to use the aliases for a a "Mailing List with a digested form of the list and archiving" when adding entries to /etc/aliases.majordomo. I've supplied my aliases in newdumm.aliases.txt for your reference.

To test the list, I sent some large messages (about 7Kb) to the list. Then the list owner (me) received the following messages:

MAJORDOMO ABORT (mj_digest)!!
open(MSG, ">/usr/local/majordomo/digests/newdumm-digest/001"): 
                                            No such file or directory

MAJORDOMO ABORT (archive)!!
shlock: '/usr/local/majordomo/lists/newdumm.archive' does not exist

That told me that the above directories had not been created. So I issued the following commands as root:

mkdir /usr/local/majordomo/digests/newdumm-digest
chown majordom /usr/local/majordomo/digests/newdumm-digest
chmod 750 /usr/local/majordomo/digests/newdumm-digest

mkdir /usr/local/majordomo/lists/newdumm.archive
chown majordom /usr/local/majordomo/lists/newdumm.archive
chmod 750 /usr/local/majordomo/lists/newdumm.archive

This creates the respective directories, sets the owner to be majordomo (no the above commands do not contain a typo; they use majordom), and modify the permission to be 750. You should read the following sections from the majordomo FAQ:

2.1 - What are the proper permissions and ownership of all Majordomo files and directories?
3.11 - How do I set up a digested version of a list?

Then sending messages to the list worked, and no errors or problems with the archive or digest. If you look in the directories listed above, you should be able to see the archive/digest as they are created.

You still need to provide subscriber access to the digest.

An existing list

OK! Now I was ready to modify my existing list. But first, see the steps I took for testing a list.. If you have a list which does provide a digest or archive, you need to modify the aliases to provide the missing sections. See newdumm.aliases.txt for an example of what is required for a digest/archive list. Then add what's missing to your list.

As an aid, here's what I removed from /etc/aliases.majordomo:

#
# below this point is the 
# original adsl mailing list
#
#owner-adsl:dan
#owner-adsl-digest:owner-adsl
#owner-adsl-outgoing:owner-adsl
#owner-adsl-digest-outgoing:owner-adsl
#adsl-approval:owner-adsl
#adsl-digest-approval:owner-adsl
#
#adsl: "|/usr/local/majordomo/wrapper resend -l adsl -h 
                                    ns.freebsddiary.cx adsl-outgoing"
#adsl-outgoing::include:/usr/local/majordomo/lists/adsl
#
#adsl-request: "|/usr/local/majordomo/wrapper majordomo -l adsl"

And here's what I added:

# Mailing List with a digested form of the list and archiving
#

owner-adsl:dan
owner-adsl-digest:owner-adsl
owner-adsl-outgoing:owner-adsl
owner-adsl-digest-outgoing:owner-adsl
adsl-approval:owner-adsl
adsl-digest-approval:owner-adsl

adsl: "|/usr/local/majordomo/wrapper resend -l adsl -h 
                             ns.freebsddiary.cx adsl-outgoing"

#
# NOTE that I changed the definition of adsl-digest from the original.
# See below for the new version.
#
#adsl-digest:adsl
adsl-outgoing::include:/usr/local/majordomo/lists/adsl,
  "|/usr/local/majordomo/wrapper archive -a -m -f 
                /usr/local/majordomo/lists/adsl.archive/adsl.archive",
  "|/usr/local/majordomo/wrapper digest -r -C -l adsl-digest 
                                             adsl-digest-outgoing"
adsl-digest-outgoing::include:/usr/local/majordomo/lists/adsl-digest

adsl-request: "|/usr/local/majordomo/wrapper majordomo -l adsl"
adsl-digest-request: "|/usr/local/majordomo/wrapper majordomo 
                                                      -l adsl-digest"

#
# here is the newly defined version of the digest.
# it has been split into two lines for readability
#
adsl-digest: "|/usr/local/majordomo/wrapper resend -l adsl-digest 
                       -h ns.freebsddiary.cx adsl-digest-outgoing"

The complete text of these example appear in samples/adsl-digest.aliases.txt (it's easier to copy/paste or fetch). Remember to run newaliases after making changes to /etc/aliases.majordomo.

Then you need to create the directories for the archive and the digest:

mkdir /usr/local/majordomo/digests/adsl-digest
chown majordom /usr/local/majordomo/digests/adsl-digest
chmod 750 /usr/local/majordomo/digests/adsl-digest

mkdir /usr/local/majordomo/lists/adsl.archive
chown majordom /usr/local/majordomo/lists/adsl.archive
chmod 750 /usr/local/majordomo/lists/adsl.archive

mkdir /usr/local/majordomo/lists/adsl-digest.archive
chown majordom /usr/local/majordomo/lists/adsl-digest.archive
chmod 750 /usr/local/majordomo/lists/adsl-digest.archive

Note that even the digest has an archive.

Providing subscriber access to the digest

In the above examples, you created a digest. The next step is to give people access to that digest. The easiest way to do this is to create another list. I called mine adsl-digest. In the previous section, I showed you the commands for creating the directories for the adsl-digest.

Testing a list

When you are testing a new list, if things go wrong, nobody is affected. But if you are testing an existing list, you can affect your users. And people get a bit upset if you send too many test messages to the list. Here's how I avoided such problems.

I shut down incoming mail by changing my firewall rules to refuse incoming connections on port 25 (smtp).
I took a backup of the mailing list files including the configuration and subscription membership.
Removed everyone from the mailing list and added a test address.

By shutting down incoming mail, I ensured that incoming mailing list requests would not interfere with my testing and they would be queued up at the sending mail servers (don't do this for very long). Then I also removed everyone from the list and put only my name in the list file. This allowed me to test without affecting others.

After my testing finished, I did this:

Restored the subscription membership to the original subscribers.
Modified my firewall rules to allow incoming mail.

If you were testing digests, you might want to reset the digest volume numbers to what they were before. Check your .config file in /usr/local/majordomo/lists:

        # digest_volume        [integer] (1) <digest>
        # The current volume number
digest_volume       =   1

Need more help on this topic? Click here
This article has no comments
Show me similar articles