Purpose of this assignment:

• To give you some experience with Perl and regular expressions.

General idea of the assignment:

Read in an HTML file and turn it into plain text by removing all tags and replacing all entities with the characters that they represent. Try to insert newlines in appropriate places.

Details:

Basically, the problem is to turn a correct HTML file into plain text.

• All tags (things within angle brackets, < >) should be removed. However, a few tags will require a bit of special processing.
  • <a href=URL ...>Link text</a> tags should be replaced by Link text [URL], that is, the link text followed by the URL in square brackets.
  • <img src=URL alt=text> tags should be replaced by text [URL] if the alt attribute is present, or just by [URL] if the alt attribute is not present.
  • Each <br> tag should be replaced by a newline, and each <p> tag by two newlines.
  • Each header (<h1>...<h6>, <title>, <div>) should be on a line by itself, with at least one blank line before and after (more than one blank line is OK).
  • Within a <pre> tag, don't run lines together or split long lines (other tags within <pre> shoule be handled normally, however)
• If at all possible, no lines should be longer than 72 characters. Lines should only be broken at blanks. Don't break lines any earlier than necessary.
• Entities should be replaced by their corresponding characters. The only entities you need to deal with are: <  >  &  "  ' and  .

You should read from STDIN and write to STDOUT.

Things to watch out for:

• The order in which you do things is important. You probably want to leave the entities until last, for example.
• All tags may have attributes (so you can't just search for <br>, for example), and the attributes may be in any order.

There are many other tags that could be handled specially. However, to do a really good job turning HTML into text is far more work than I want for this assignment, so if your output isn't perfect for every possible HTML file, don't worry about it. If you can do as much as I've asked for, you've learned enough Perl for this course!

Getting Perl:

Linux in Moore 207: ActivePerl is installed in Moore 20. To run it, create a Perl file (with extension .pl), say, test.pl. Open an X Terminal window in the directory with your program, and type perl test.pl. The initial line, #!/usr/bin/perl, is accepted (if correct) but not required.

Windows: Go to http://www.activestate.com/Products/ActivePerl/ and click the DOWNLOAD link at the very bottom of the page. Download ActivePerl 5.8.4, Windows, MSI. When downloaded, double-click the file ActivePerl-5.8.4.810-MSWin32-x86.msi and just accept all the defaults. To run it, create a Perl file (with extension .pl), say, test.pl. Open a DOS (cmd) window in the directory with your program, and type perl test.pl. The initial line, #!/usr/bin/perl, if present, appears to be ignored. (If you really want more detail than this, see http://www.extropia.com/tutorials/winperl/).

Macintosh OS X: You already have Perl (of course!). Open a Terminal window, create a Perl file (say, test), and type perl test. The initial line, #!/usr/bin/perl, if present, appears to be ignored.

Due date:

Tuesday, November 23, before midnight.