Chapter 14: Keep what you find

Search and throw away

To novices, everything is difficult. During their first trips online, they may feel as if moving to the other side of the globe. There, they would meet other newspapers, magazines, information sources, and services. It takes time to find your way around in this new environment.

To find online gold mines, trial and error are necessary. As you get more experience, however, focus tends to shift from getting information to digesting. Retrieving the data is 'in your fingers', and doesn't bother much anymore.

The bad news is that your reading speed remains at the same old level, while the amount of retrieved data grows. Most users therefore need a strategy for handling the resource. This is my take:

• In our time, when people tend to talk more than they listen, we'll usually find more information than knowledge. Therefore, say NO to irrelevant information! It is rarely worth keeping.
• There is no good reason to learn things that you really do not need to know. Practice "selective ignorance."
• Regularly rate your online sources. Critically. Discard those costing you more time, money and effort than they are worth. Concentrate on those giving the best returns.
• Regularly adjust the frequency with which you visit selected services to match their usefulness. What used to be daily visits, may have to be downgraded to once per week or month. Consider replacing daily news monitoring by clipping services.

Dig for 'overview'

Plan 'overview' and 'digging for details' with different frequencies. 'Overview' refers to online trips made to get an impression of what generally goes on. An example:

Using Agora World Wide Web by email servers, it is easy to request a list of articles from Usenet newsgroups at preset intervals. For example, email programs like Eudora Pro let you have the same message scheduled to be sent at a certain time and date every month. The trick is explained in Chapter 12 under Usenet by email.

Digging for useful information in today's news may also be done by Agora mail coupled with a system or feature that let the web address (URL) be sent at desireable intervals. You may want to check out NetMind in Chapter 11 for the job if your email program is unable to do it.

On one workstation, I use a shareware Windows utility called BarClock to warn me once per day. "You should send your Agora mail now!" The warning flashes up on my screen in such a way that it cannot be ignored. On another workstation, a MS-DOS based robot does it automatically for me. No manual intervention required.

Say you want to monitor what goes on in the alt.winsock newsgroup on a monthly basis. Prepare a monthly message containing the following:

  To: agora@dna.affrc.go.jp
  Cc: 
  Subject: 
  Text: send news:alt.winsock

Each 30 days, you should now receive a mail giving you a list of topics discussed in the newsgroup. It would look something like this:

  Date: Mon, 16 Sep 1996 05:38:57 -0500 (CDT) 
  From: agora@dna.affrc.go.jp 
  Subject: Document requested (was:)(URL: news:alt.winsock)
 
               Newsgroup alt.winsock,  Articles 86509-86528
   (Earlier articles[1]...) Articles in alt.winsock 
 
 "Re:Error Code 16"- Phil Joy[2] 
 "Re: Netscape Mail"- Schaft[3] 
 "Re: OFFLINE NEWSGROUP READER ??"- djl[4] 
 "Re: OFFLINE NEWSGROUP READER ??"- Matthew McDonald[5] 
 "Re: OFFLINE NEWSGROUP READER ??"- Derek Woodlands[6] 
 "Re: OFFLINE NEWSGROUP READER ??"- Matthew McDonald[7] 
 "Re: AOL coexistence, mcs.support"- Leonard Grossman[8] 
 "Re: Netscape Mail"- David Woolley[9] 
 "MAKE 50 K IN 1 MONTH"- The One[10] 
 "Re: OFFLINE NEWSGROUP READER ??"- Melanie Ley[11] 
 "Re: *** ANNOUNCE: new W95 FTP beta ***"- [12] 
 "Re: Win 95 FTP Server"- [13] 
 "selecting multiple items with mouse instead of ctrl"-[14]
 "Re: Win 95 FTP Server"- Alun Jones[15] 
 "free agent purge thinger"- [16] 
 "help"- w4100@epix.net[17] 
 "Re: Win 95 FTP Server"- [18] 
 "Re: AOL coexistence, mcs.support"- Randy Hubbard[19] 
 "Upgrading win95 winsock"- boomer[20] 
 "I have no idea what i'm doing"- mal4273@tamu.edu[21] 
 
   ___________________________________ 
   Post to alt.winsock[22] 
 
 *** References from this document *** 
 [orig] news:alt.winsock 
 [1] news:alt.winsock/86489-86508 
 [2] news:51g25t$g1l@news.internetmci.com 
 [3] news:51g24d$ora@nntp1.best.com 
 [4] news:323bbaae.105680122@news.aimnet.com 
 [5] news:323bcd4a.8728606@news.netspace.net.au 
 [6] news:323be00d.2515074@news.mt.net.au 
 [7] news:323bf639.6886883@news.netspace.net.au 
 [8] news:51gtbh$61o@nntp1.mcs.net 
 [9] news:DxqqEG.wq@djwhome.demon.co.uk 
 [10] news:323C0C0C.63F4@cyberway.com.sg 
 [11] news:32404693.99526427@nntp.crl.com 
 [12] news:51hltp$gfp@tetsuo.communique.net 
 [13] news:51hls1$gfp@tetsuo.communique.net 
 [14] news:51i3gi$bbg@tetsuo.communique.net 
 [15] news:51i7iu$100_002@alun.eden.com 
 [16] news:51i6n2$10s1@tetsuo.communique.net 
 [17] news:323CC90B.3BE7@epix.net 
 [18] news:51i8km$15eb@tetsuo.communique.net 
 [19] news:323CAADC.A2@wwnet.com 
 [20] news:323C9F2E.5412@sprynet.com 
 [21] news:51imni$t4v@news.tamu.edu 
 [22] newspost:alt.winsock

Browsing the index will give you an idea about what's on the members minds for the moment. If you like, you can also request individual articles from the list.

Say you want to check out the topic "OFFLINE NEWSGROUP READER ??" To do so, send a mail to the Agora server containing the following commands:

 send news:323bbaae.105680122@news.aimnet.com 
 send news:323bcd4a.8728606@news.netspace.net.au 
 send news:323be00d.2515074@news.mt.net.au 
 send news:323bf639.6886883@news.netspace.net.au

Note: Some Agora servers let you send for these articles by returning item numbers rather than the full addresses.

One of these articles came back with the following text (abbreviated):

  Subject: 
   (URL: news:323bf639.6886883@news.netspace.net.au)
   Reply to  matthew@sv.net.au (Matthew McDonald)[1]
   RE: OFFLINE NEWSGROUP READER ?? 
   Sun, 15 Sep 1996 12:27:59 GMT
 
  kingpin@mail.mt.net.au (Derek Woodlands) wrote: 
 
  >On Fri, 13 Sep 1996 22:11:37 GMT, nephite@worldaccess.com
  >Jacob Proffitt) wrote: 
  > 
  >>I'm really pleased with Agent, the commercial version
  >>of Free Agent. FA is also a good app, and it is free.8-)
  >> 
  >>I agree. Free Agent is a good way to get a feel for it.
  >>Except that it doesn't handle MIME. Nor does Agent .99
   
  The next release of Agent, due any time now will support 
  MIME..
 
  Regards 
  Matthew

The message also contained 14 references to the other articles in the topic thread, to the author, and other newsgroups where the message had appeared.

Several communication programs and offline readers have built- in quick scan features. For example, TAPCIS does this in CompuServe forums.

When retrieving conference messages from bulletin boards using 1stReader at high speed, then the cost of retrieving all new items may be insignificant. You might as well do it. When reading captured mail, 1stReader lets you select messages to read from a list of subjects. You can save what you want to keep, and delete the rest.

By regular scanning subject headers, you reduce the risk of missing important trends, for example because authors were using other terms on the subject line than you expected.

Scanning also lets us discover if the discussion is heading off in other, interesting directions.

After a while, you'll have a set of sources, persons, and tools that will provide you with what you need. This is your personal infrastructure of electronic information. Now, you must maintain and cultivate it.

Chapter 12 contains hints about how to search World Wide Web indexes like Lycos and Alta Vista by email. Use this method regularly to check what is being said about yourself, your company, a technology, a hobby, whatever.

Filtering incoming information

Chances are that you will retrieve more information than you can read. Sometimes it takes weeks or even months for me to get up to date with all my electronic readings.

Several email programs can automatically filter retrieved information in folders, for example based on information found in the From or To-line of incoming mail. This feature is very helpful!

Example: I get a lot of mail. Upon receipt, messages are automatically filtered into the following groups (folders):

Urgent mail		All mail coming from a predefined list of senders.
General mail		All mail that was not captured by my filtering commands.
Kidlink		Selected conference mail from a defined list of Kidlink mailing lists.
Errors		Error messages received by mailing lists that I manage (the TOW announcement list for readers of   the Online World handbook is one of these lists. At listserv@listserv.nodak.edu).
Browse		Information that is less time critical, and that it is enough to browse for interesting information.
Read		Information that I want to read once I get the time. (If I don't get the time, it still there on my disk until deleted, and I can therefore search it's contents.)
Outgoing		Archive of all sent mail.

Periodically, my life is so hectic that I just have time for my Urgent mail. Whenever this happens, it is very useful to have them filtered out and stored in one place.

Example: Daily, Web by email services (see Chapter 12) provides me with lists of available news stories on the net. One of my sources is NewsLinx (at http://www.newslinx.com/).

I'm supposed to read through these story menus, select what I want to read, and go fetch them. However, I rarely have time to do this, so I've asked the Eudora Light email program to do part of the job for me.

Several filters are set to search for "newslinx.com" on the incoming mails' subject line. Other settings initiate a search in selected messages to see if there are stories on desired topics.

One filter look for the word "newspaper" in NewsLinx menus. Whenever the word is found, the incoming message is marked as High Priority, and the subject line changed to say "Keyword newspaper found." This helps me avoid missing useful articles on interesting topics.

Post-processing the data

Your data capture is completed, and the retrieved data is stored on the hard disk in more or several files. Your next task is to

• Read the received material.
• Cut and paste selected parts to archives or work files.
• Prepare responses to electronic mail.
• When done, delete all temporary files.

Some people just delete all mail they don't want, and let the rest remain intact in their original folders. Most mail folders are easy to search with tools like AltaVista Discovery and Lookfor (see below), so this may be a solution for them.

Personally, I get too much mail for that. When I read it, my preferred method is to cut out what I need, and store the texts without the lengthy mailer headers in a more systematic way. I do this to reduce disk storage requirements, and speed up searches.

My mail is stored in files and directories using names that show the source of the information. This makes it quick to find where to search. A set of macro commands make cutting and appending mail easy.

Often, I use the year, or a month/year code, in the file name extension. For example, the file TOW.96 contains messages retrieved from the TOW mailing list during 1996.

Some advanced email programs have these features built in. With Eudora Light it is easy. Click on Transfer, and dump the message into a mailbox (or folder) of your choice.

You may also use your favorite word processor, or something else. There are several alternatives. Personally, I use the LIST utility in my system.

LIST is my favorite MS-DOS shareware file viewer program. You can retrieve it from the Internet, and most bulletin boards. Using LIST, it is difficult to destroy your precious retrieved data while reading, cutting and pasting.

MORE ABOUT LIST: Assume that you want to postprocess Eudora's IN.MBX file, and that you are using the file name convention suggested above. Type LIST and press ENTER. A list of file names in your Eudora directory will appear on your screen. Press S to sort the list, D to have them sorted by creation date. The newest files are at the bottom of the list.

Move the cursor (using the Arrow keys) to the input file you want to read, and press ENTER. Scroll up and down in IN.MBX by pressing the PgUp/PgDn or the arrow keys.

On your screen is a piece of information that you want to keep for future reference. Mark the text with ALT-M commands (keep the ALT key pressed down, while pressing M), and then ALT-D. LIST will ask you for a file name. If the information is from the TOW mailing list, and the year is 1996, enter TOW.96, and the text is appended to what is already there.

This method allows you quickly to mark and append parts of your input file to various archive files. Press ESC to return to the file list when through. If you press D, LIST asks if you really want to delete the file. Press Y, and IN.MBX is gone. - However, you'd better use Eudora for deleting mail!

LIST lets you find information stored in your archives (string search). What you find can be marked and copied to a work file. It can also be set to invoke an editor or a word processor for the selected file.

Reuse data on your hard disk

Over time your personal archives will grow in size. You begin to experience the benefits of having all this information on your hard disk. Yesterday's news is today's history, and may be used in interesting ways.

One business executive regularly monitors key technologies, customers, competitors, and suppliers. He does it by tapping sources like Kompass, Associated Press, and Reuters. Interesting bits of information are regularly stored on his disk.

Tomorrow, there is an important meeting with a major customer. First, a quick search through the personal customer database to be reminded of important events since the last meeting. An unfamiliar person is also going to be present. Maybe there is some background information, for example about a recent promotion.

Next, a quick check on major competitors. Maybe they are up to something that he needs to know about.

With efficient tools for searching your hard disk, finding information takes only a few seconds. If you are still left with questions, go online to complement.

Several modern email programs have built-in search features that can do more than search invidually named files (as in mailboxes) or messages. On MS-DOS/Windows computers, you can search your files with WordPerfect, Word, LIST, the DOS utility FIND, and a long list of other programs.

Personally, I prefer programs that let me search for more than one word at the time, like in HYDRO AND PETROCHEMICAL AND CONTRACT, or EXXON OR MOBIL.

MY FAVORITE Windows 95/98 search program is AltaVista Discovery (http://discovery.altavista.com). Not only does it let me perform complex searches through my megabytes of in- and outgoing email messages, it also lets me search my local web pages (including those in my cache), Word documents, Excel spreadsheets, and more.

MY FAVORITE MS-DOS search utility is the Lookfor shareware program. It is small, fast, and is superior for searches in text files. You can store your finds in work files, or print them out on paper. LOOKFOR is not an indexing program. It is ready to search anywhere, anytime. Retrieve it at http://home.eunet.no/~presno/bok/lookfr50.zip .

Discipline and organization is required to get the most out of your file archives. You must decide what to do with each piece of information: Should it be printed out and be read in front of the fireplace in the evening, or should it be circulated? Should it be stored on your disk, or be refined before storage?

Use standard file names that are easy to remember. If not, risk having to view files to find out what they contain.

It may take longer to find a piece of information in a casual file on a large disk, than look up a piece of information on paper in your inbox. Therefore, finish handling your capture file while you read it on your screen:

Send the pieces to their final destination. Make immediate transfers to TO-DO files. Give the original file a name that makes it easier to move later. Have a procedure that prevents duplication of effort.

With Eudora, simply transfer messages requiring action to a mailbox named TO-DO. When the task is completed, you can transfer the letter to its final destination.

Desinformation, deception and errors

Always use several sources of information! There are people out there writing to lead you astray, in articles, forum postings, private emails, and web pages.

In early 1999, a bogus Bloomberg News story claimed that ECI Telecom Ltd. in Israel was about to purchase PairGain Technologies in the US. The story mimicked the look of Bloomberg's World Wide Web site in order to pass it off as the real thing. Many believed it to be true, and the "news" promptly drove the value of PairGain's shares up to US$200 million.

PairGain's stock price rose by as much as 32 percent, before Bloomberg announced that the story was fake, and PairGain denied any knowledge of the deal.

Often, rumors start with a private message mentioning a theory or a suspicion. After having been copied, and copied again, it can get more exposure than you get at a broadcast station, and some will even start considering it as "truth."

If you want to learn more about the "truths of the net," visit the Urban Legends page (http://www.urbanlegends.com/). The alt.folklore.urban newsgroup is devoted to the discussion and debunking of urban legends and related issues. The NewsWatch site (http://www.newswatch.org/) is another interesting place to visit. It is operated by The Center for Media and Public Affairs, a nonpartisan research organization that conducts scientific analyses of the media. They promise to "tell you when [news is] accurate, when it's wrong, and why."

Without clear, convincing evidence to appease doubting minds, conspiracy theories abound, sometimes with good cause, frequently with little substance.

Stock promoters and con artists have spread rumors to pump up a stock's price and generate quick profits at the expense of gullible investors. The Internet is a place where investors should be wary of stock scams.

Desinformation hurts everybody and comes from all sides. Even professional news agencies, like Associated Press, Reuters and Agence France-Presse, regularly stumble.

Most news is written by journalists reporting what they have seen, read or heard. Their interpretation of the situation may be wrong. Supplement your news with what knowledgeable people say (by email or in conferences), when knowing the facts is important.

Another hint: Errors will occasionally be discovered and reported by the news sources, but always after the fact. Always store these reports in your archives, and make it a rule to search to the end when looking for something. Otherwise, you may never discover the corrections.

Links for warnings and more information:

• http://ciac.llnl.gov/ciac/CIACChainLetters.html
• http://ciac.llnl.gov/ciac/CIACHoaxes.html
• http://www.kumite.com/myths/
• http://urbanlegends.miningco.com/

So, how do we evaluate the quality of net information? For an introduction, take a look at the Information Quality on The Internet page (http://www.virtualchase.com/govdoc/verify.shtml). You'll also find some useful hints in the Practical Steps in Evaluating Internet Resources page at http://milton.mse.jhu.edu/research/education/practical.html.