4. Setting up CNews + NNTPd

4.1. Getting the sources and stuff

4.2. Compiling and installing

For installing, first make sure you have an entry for a user called news in your /etc/password file. This is setting the news-database owner to news. Now download the source from us and untar it in the home directory of news. This creates two main directories viz. c-news and nntp. To install and compile, run the script build.sh as root in the directory that contains the script. It is important that the script run as root as it sets ownerships, installs and compiles the source as user news. This is a one-step process that puts in place both the C-News and the NNTP software, setting correct permissions and paths. Following is a brief description of what build.sh does:

• Checks for the OS platform and exits if it is not Linux.

• Again, exits if you are not running as root.

• Looks for and exits if cannot find the above two directories.

• Compiles C-News and performs regression tests if the compilation was successfull. Sends out a warning to read the error file make.out.r and to fix 'em. Compilation erros are written to a file called make.out.

• Performs the above operation in the nntp directory, too.

• Checks for the presence of the three key directories: $NEWSARTS - (/var/spool/news) that houses the artciles, $NEWSCTL -(/var/lib/news) that contain configuration, log and status files and $NEWSBIN - (/usr/lib/newsbin) that contain binaries and executables for the working of the Usenet News system. Tries to create them if non-existent and exits if it results in failure.

• Changes the ownership of these directories to news.news. This is important since the entire Usenet News System runs as user news. It will not function properly as any other user.

• Then starts the installation process of C News. It runs make install to install binaries at the right locations; make setup to set the correct paths and umask, create directories for newsgroups, determine who will receive reports; make ui to set up inews and injnews and make readpostcheck to use readnews, postnews and checknews scripts provided by C News. The errors, if any are to be found in the respective make.out files. e.g. make.setup will write errors to make.out.setup

• Newsspool, which queues incoming batches in $NEWSARTS/in.coming directory should run as set-userid and set-groupid. This is done.

• A softlink is made to /var/lib/news from /usr/lib/news.

• The NNTP software is installed.

• Sets up the manpages for C News and makes it world readable. The NNTP manpages get installed when the software is installed. Compiles the C News documentation guide.ps and makes it readable and available in /usr/doc/packages/news or /usr/doc/news.

• Checks for the PGP binary and asks the administrator to get it, if not found.

4.3. Configuring the system: What and how to configure files?

Once installed, you have to now configure the system to accept feeds and batch them for your neighbours. You will have to do the following:

• nntpd: Copy the compiled nntpd into a directory where executables are kept and activate it. It runs on port 119 as a daemon through inetd unless you have compiled it as stand-alone. An entry in the /etc/services file for nntp would look like this:

  nntp 119/tcp \# Network News Transfer Protocol

  An entry in the inetd.conf file will be:

  nntp stream tcp nowait news path-to-tcpd path-to-nntpd

  The last two fields in the inetd.conf file are paths to binaries of the tcp and the nntp daemon respectively.

• Configuring control files: There are plenty of control files in $NEWSCTL that will need to be configured before you can start using the news system. The files mentioned here are also discussed in the first section of the section titled "Components of a running system>". These control files are dealt in detail in the following below.

  • sys: One line per system/NDN listing all the newsgroup hierarchies each system subscribes to. Each line is prefixed with the system name and the one beginning with

    ME:

    indicates what your server is willing to receive. Following are typical entries that go into this file:

    ME:comp,news,misc,netscape

    This line indicates what newsgroups your server has subscribed to.

    server/server.starcomsoftware.com:all,!general/all:f

    This is a list of newsgroups your server will pass on to your NDN. The newsgroups specified should be a comma separated list and the entire line should contain no spaces. The f flag indicates that the newsgroup name and the article number alongwith its size will make up one entry in the togo file in the $NEWSARTS/out.going directory.

  • explist: This file has entries indicating which articles expire and when and whether they have to be archived. The order in which the newsgroups are listed is important. An example follows:

    comp.lang.java.3d x 60 /var/spool/news/Archive

    This means that the articles of comp.lang.java expire after 60 days and shall be archived in the directory mentioned in the fourth field. Archiving is an option. The second field indicates that this line applies to both moderated and unmoderted newsgroups. m would specify moderated and u would specify unmoderated groups. If you want to specify an extremely large no. as the expiry period you can use the keyword "never".

  • batchparms: sendbatches is a program that administers batched transmission of news articles to other sites. To do this it consults the batchparms file. Each line in the file specifies the behaviour for each of your NDN mentioned in the sys file. There are five fields for each site to be specified.

    server u 100000 100 batcher | gzip -9 | viauux -d gunzip

    The first field is the site name which matches the entry in the sys file and has a corresponding directory in $NEWSARTS/out.going by that name.

    The second field is the class of the site,u for UUCP and n for NNTP feeds. A "!" in this field means that batching for this site has been disabled.

    The third field is the size of batches to be prepared in bytes.

    The fourth field is the maximum length of the output queue for transmission to that site.

    The fifth field is the command line to be used to build, compress and transmit batches to that site. The contents of the togo file are made available on standard input.

  • controlperm: This file controls how the news system responds to control messages. Each line consists of 4-5 fields separated by white space. Control messages has been discussed in "Section 2.4>".

    comp,sci tale@uunet.uu.net nrc pv news.announce.newsgroups

    The first field is a newsgroup pattern to which the line applies.

    The second field is either the keyword "any" or an e-mail address. The latter specifies that the line applies to control messages from only that author.

    The third field is a set of opcode letters indicating what control operations need to be performed on messages emanating from the e-mail address mentioned in the second field. n stands for creating a newgroup, r stands for deleting a newsgroup and c stands for checkgroup.

    The fourth field is a set of flag letters indicating how to respond to a control message that meets all the applicability tests:

    y Do it. n Don't do it. v Report it and include the entire control message in the report. q Don't report it. p Do it iff the control message carries a valid PGP signature.

    Exactly one of y, n or p must be present.

    The fifth field, which is optional, will be used if the fourth field contains a p. It must contain the PGP key ID of the public key to be used for signature verification.

  • mailpaths: This file describes how to reach the moderators of various hierarchies of newsgroups by mail. Each line consists of two fields: a news group pattern and an e-mail address. The first line whose group pattern matches the newsgroup is used. As an example:

    comp.lang.java.3d somebody@mydomain.com all %s@moderators.uu.net

    In the second example, the %s gets replaced with the groupname and all dots appearing in the newsgroup name are substituted with dashes.

  • Miscellaneous files: The other files to be modified are:

    • mailname: Contains the Internet domain name of the news system. Consider getting one if you don't have it.

    • organization: Contains the default value for the Organization: header for postings originating locally.

    • whoami: Contains the name of the news system. This is the site name used in the Path: headers and hence should concur with the names your neighbours use in their sys files.

  • active file: This file specifies one line for each newsgroup (not just the hierarchy) to be found on your news system. You will have to get the most recent copy of the active file from ftp://ftp.isc.org/usenet/CONFIG/active and prune it to delete newsgroups that you have not subscribed to. Run the script addgroup for each newsgroup in this file which will create relevant directories in the $NEWSARTS area. The addgroup script takes two paramters: the newsgroup name being created and a flag. The flag can be any one of the following:

    y local postings are allowed n no local postings, only remote ones m postings to this group must be approved by the moderator j articles in this group are only passed and not kept x posting to this newsgroup is disallowed =foo.bar articles are locally filed in "foo.bar" group

    An entry in this file looks like this:

    comp.lang.java.3d 0000003716 01346 m

    The first field is the name of the newsgroup. The second field is the highest article number that has been used in that newsgroup. The third field is the lowest article number in the group. The fourth field is a flag as explained above.

  • newsgroups file: This contains a one-line description of each newsgroup to be found in the active file. You will have to get the most recent file from ftp://ftp.isc.org/usenet/CONFIG/newsgroups and prune it to remove unwanted information. As an example:

    comp.lang.java.3d 3D Graphics APIs for the Java language

  • Aliases: These aliases are required for trouble reporting. Once the system is in place and scripts are run, anomalies/problems are reported to addresses in the /etc/aliases file. These entries include email addresses for newsmaster, newscrisis, news, usenet, newsmap. They should ideally point to an email address that will be accessed at regularly. Arrange the emails for newsmap to be discarded to minimize the effect of sendsys bombing by practical jokers.

  • Cron jobs: Certain scripts like newsrun that picks up incoming batches and maintenance scripts, should run through news-database owner's cron which is news. The cron entries ideally will be for the following: A more detailed report can be found in "Section 9.4>"

    1. newsrun: This script processes incoming batches of article. Run this as frequently as you want them to get digested.

    2. sendbatches: This script transmit batches to the NDNs. Set the frequency according to your requirements.

    3. newsdaily: This should be run ideally once a day since it reports errors and anomalies in the news system.

    4. newswatch: This looks for errors/anomalies at a more detailed level and hence should be run atleast once every hour

    5. doexpire: This script expires old articles as determined by the explist file. Run this once a day.

  • newslog: Make an entry in the system's syslog.conf file for logging messages spewed out by nntpd in newslog . It should be located in $NEWSCTL. The entry will look like this:

    news.debug -/var/lib/news/newslog

  • Newsboot: Have this run (as news the news-database owner) when the system boots to clear out debris left around by crashes.

  • Add a Usenet mailer in sendmail: The mail2news program provided as part of the source code is a handy tool to send an e-mail to a newsgroup which gets digested as an article. You will have to add the following ruleset and mailer definition in your sendmail.cf file:

    • Under SParse1, add the following:

      R$+ . USENET < @ $=w . > $#usenet $: $1

    • Under mailer definitions, define the mailer Usenet as:

      MUsenet P=/usr/lib/newsbin/mail2news/m2nmailer, F=lsDFMmn, S=10, R=0, M=2000000, T=X-Usenet/X-Usenet/X-Unix, A=m2nmailer $u

    In order to send a mail to a newsgroup you will now have to suffix the newsgroup name with usenet i.e. your To: header will look like this:

    To: misc.test.usenet@yourdomain.

    The mailer definition of usenet will intercept this mail and post it to the respective newsgroup, in this case, misc.test

  This, more or less, completes the configuration part.

4.4. Testing the system

To locally test the system, follow the steps given below:

• post an article: Create a local newsgroup

  cnewsdo addgroup mysite.test y

  and using postnews post an article to it.

• Has it arrived in $NEWSARTS/in.coming?: The article should show up in the directory mentioned. Note the nomenclature of the article.

• When newsrun runs: When newsrun runs from cron , the article disappears from in.coming directory and appears in $NEWSARTS/mysite/test. Look how the newsgroup, active, log and history (not the errorlog) files and .overview file in $NEWSARTS/mysite/test reflect the digestion of the file into the news system.

• reading the article: Try to read the article through readnews or any news client. If you are able to, then you have set most everything right.

4.5. pgpverify and controlperms

As mentioned in "Section 2.4>", it becomes necessary to authenticate control messages to protect yourself from being attacked by pranksters. For this, you will have to configure the $NEWSCTL/controlperm file to declare whose control messages you are willing to honour and for what newsgroups alongwith their public key ID. The controlperm manpage shall give you details on the format.

This will work only in association with pgpverify which verifies the Usenet control messages that have been signed using the signcontrol process. The script can be found at ftp://ftp.isc.org/pub/pgpcontrol/pgpverify. pgpverify internally uses the PGP binary which will have to be made available in the default executables directory. If you wish to send control messages for your local news system, you will have to digitally sign them using the above mentioned signcontrol program which is available at ftp://ftp.isc.org/pub/pgpcontrol/signcontrol. You will also have to configure the signcontrol program accordingly.

4.6. Feeding off an upstream neighbour

For external feeds, commercial customers will have to buy them from a regular News Provider like dejanews.com or newsfeeds.com. You will have to specify to them what hierarchies you want and decide on the mode of transmission, i.e. UUCP or NNTP, based on your requirements. Once that is done, you will have to ask them to initiate feeds, and check $NEWSARTS/in.coming directory to see if feeds are coming in.

If your organisation belongs to the academic community or is otherwise lucky enough to have an NDN server somewhere which is willing to provide you a free newsfeed, then the payment issue goes out of the picture, but the rest of the technical requirements remain the same.

One problem with incoming NNTP feeds is that it is far easier to use (relatively) efficient NNTP inflows if you have a server with a permanent Internet connection and a fixed IP address. If you are a small office with a dialup Internet connection, this may not be possible. In that case, the only way to get incoming newsfeeds by NNTP may be by using a highly inefficient pull feed.

4.7. Configuring outgoing feeds

If you are a leaf node, you will only have to send feeds back to your news provider for your postings in public newsgroups to propagate to the outside world. To enable this, you need one line in the sys and batchparms files and one directory in $NEWSARTS/out.going. If you are willing to transmit articles to your neighbouring sites, you will have to configure sys and batchparms with more entries. The number of directories in $NEWSARTS/out.going shall increase, too. Refer to first two sections of the chapter titled "Components of a running system>"for a better understanding of outgoing feeds. Again, you will have to determine how you wish to transmit the feed: UUCP or NNTP.