Purpose of this assignment:

• To familiarize you with regular expressions.

General idea of the assignment:

Write a Java application to read in an HTML page, remove all the HTML tags from it, and display the result in a large scrollable JTextArea.

Details:

Use a Swing GUI dialog box to enter a URL into your program. I recommend a JOptionPane as the quickest and easiest way to get the URL (see http://java.sun.com/docs/books/tutorial/uiswing/components/dialog.html for a tutorial). Note that you need to type a complete URL (typically starting with http://) into the dialog box.

Go to that URL and get the HTML file that is there; read it as text. See the example at TryURL.java to get the necessary combination of I/O calls. (Note: A 403 return code means that they don't allow you to connect this way--http://news.google.com/ is an example--just find a different site.)

Most tags should just be removed, but with the following exceptions:

• Any text in one of the containers <title>, <div>, <h1>, <h2>, <h3>, <h4>, <h5>, or <h6> should be put on a line by itself, with a single blank line before and after it.
  
  
• A <p> tag results in a blank line. (Do not assume that there will be a matching </p> end tag--few people actually use them.) Try to avoid multiple consecutive blank lines.
  
  
• A <br> tag should be replaced by a newline.
  
  
• After a <ul>, every <li> should start a new line, beginning with " * ".
  
  
• After an <ol>, every <li> should start a new, numbered line, with the first <li> starting at 1. Counting ends when the matching </ol> is encountered.
  
  
• An <img> tag with an alt="text" attribute should be replaced by [image: text]. If it has no alt attribute, it should be replaced by just [image].

Use regular expressions to do most of the work of finding and removing or replacing tags.

• Tags are not case sensitive: ImG is the same as img.
  
  
• Tags may contain whitespace, for example <img src = "foo.gif"  >.
  
  
• Start tags may contain attributes, even meaningless attributes; for example, <p stuff="nonsense">.
  
  
• Tags may extend over more than a single line.
  
  
• The value of an attribute may be (1) enclosed in double-quote marks, (2) enclosed in single-quote marks, (3) terminated by whitespace, or (4) terminated by the ">" of the start tag.

We will test your program with both simple tags and with tags containing extra whitespace, tags that are split across lines, etc.

Finally:

• HTML pages are seldom perfect, so there's no point in checking for errors. Minor problems in the HTML may result in minor problems in your extracted text, and that's okay. Minor problems in the HTML should not cause major problems in the output, however; your program should still muddle through somehow.
  
  
• Save yourself a lot of grief. Use many smaller regular expressions rather than a few long ones.
  
  
• Your program doesn't have to be fast. However, it shouldn't be mind-numbingly slow, either. It's generally a bad idea to use regular expressions on very long strings (such as an entire page). The general rule is: Make it work before you make it faster.

Due date:

Please turn in your program via Blackboard before midnight, Wednesday October 10 Friday October 12.