danbikel.lisp
Class SexpTokenizer

java.lang.Object
  danbikel.lisp.WordTokenizer
      danbikel.lisp.SexpTokenizer

public class SexpTokenizer
extends WordTokenizer

A class for tokenizing simple S-expressions, where there are only strings delimited by whitespace or parentheses (as implemented by WordTokenizer). Comments are lines where the first non-whitespace character is a semicolon (the character ';').

Field Summary

Fields inherited from class danbikel.lisp.WordTokenizer

javadocHack, sval, ttype

Constructor Summary

SexpTokenizer(File file, String encoding, int bufSize)
Convenience constructor, creating a SexpTokenizer around a BufferedReader around a FileInputStream.

SexpTokenizer(File file, String encoding, int bufSize, boolean comments)
Convenience constructor, creating a SexpTokenizer around a BufferedReader around a FileInputStream.

SexpTokenizer(InputStream stream, String encoding, int bufSize)
Convenience constructor, creating a SexpTokenizer around a BufferedReader around a InputStreamReader.

SexpTokenizer(InputStream stream, String encoding, int bufSize, boolean comments)
Convenience constructor, creating a SexpTokenizer around a BufferedReader around a InputStreamReader.

SexpTokenizer(Reader inStream)
Constructs a SexpTokenizer with the specified stream and the default comment-recognition option, which is true.

SexpTokenizer(Reader inStream, boolean comments)
Constructs a SexpTokenizer with the specified stream and comment-recognition option.

SexpTokenizer(Reader inStream, boolean comments, char[] ordinary)
Constructs a SexpTokenizer with the specified stream, comment-recognition option and set of ordinary characters, which are actually those characters to be treated as metacharacters (i.e., not part of tokens, but delimiters of tokens).

SexpTokenizer(String filename, String encoding, int bufSize)
Convenience constructor, creating a SexpTokenizer around a BufferedReader around a FileInputStream.

SexpTokenizer(String filename, String encoding, int bufSize, boolean comments)
Convenience constructor, creating a SexpTokenizer around a BufferedReader around a FileInputStream.

SexpTokenizer(URL url, String encoding, int bufSize)
Convenience constructor, creating a SexpTokenizer around a BufferedReader around an InputStreamReader around the stream created by calling url.openStream().

SexpTokenizer(URL url, String encoding, int bufSize, boolean comments)
Convenience constructor, creating a SexpTokenizer around a BufferedReader around a InputStreamReader around the stream created by calling url.openStream().

Method Summary

Methods inherited from class danbikel.lisp.WordTokenizer

close, commentChar, lineno, nextToken, numCharsRead, ordinaryChar, ordinaryChars, pushBack

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SexpTokenizer

public SexpTokenizer(Reader inStream,
                     boolean comments)

Constructs a SexpTokenizer with the specified stream and comment-recognition option.

Parameters:

inStream - the input stream for this tokenizer

comments - indicates whether to recognizes comment lines

See Also:

WordTokenizer.commentChar(int)

SexpTokenizer

public SexpTokenizer(Reader inStream,
                     boolean comments,
                     char[] ordinary)

Constructs a SexpTokenizer with the specified stream, comment-recognition option and set of ordinary characters, which are actually those characters to be treated as metacharacters (i.e., not part of tokens, but delimiters of tokens).

Parameters:

inStream - the input stream for this tokenizer

comments - indicates whether to recognizes comment lines

ordinary - an array of all characters to be treated as metacharacters

See Also:

WordTokenizer.ordinaryChar(char), WordTokenizer.commentChar(int)

SexpTokenizer

public SexpTokenizer(Reader inStream)

Constructs a SexpTokenizer with the specified stream and the default comment-recognition option, which is true.

Parameters:

inStream - the input stream for this tokenizer

SexpTokenizer

public SexpTokenizer(String filename,
                     String encoding,
                     int bufSize)
              throws UnsupportedEncodingException,
                     FileNotFoundException,
                     IOException

Convenience constructor, creating a SexpTokenizer around a BufferedReader around a FileInputStream. The tokenizer will recognize comment lines.

Parameters:

filename - the name of the file containing S-expressions

encoding - the encoding of the file to be tokenized

bufSize - the size of the buffer of the BufferedReader that will be created

Throws:

UnsupportedEncodingException - if the specified encoding is not supported

FileNotFoundException - if the specified file does not exist

IOException

SexpTokenizer

public SexpTokenizer(String filename,
                     String encoding,
                     int bufSize,
                     boolean comments)
              throws UnsupportedEncodingException,
                     FileNotFoundException,
                     IOException

Convenience constructor, creating a SexpTokenizer around a BufferedReader around a FileInputStream.

Parameters:

filename - the name of the file containing S-expressions

encoding - the encoding of the file to be tokenized

bufSize - the size of the buffer of the BufferedReader that will be created

comments - whether this tokenizer will recognize comments

Throws:

UnsupportedEncodingException - if the specified encoding is not supported

FileNotFoundException - if the specified file does not exist

IOException

SexpTokenizer

public SexpTokenizer(File file,
                     String encoding,
                     int bufSize)
              throws UnsupportedEncodingException,
                     FileNotFoundException,
                     IOException

Convenience constructor, creating a SexpTokenizer around a BufferedReader around a FileInputStream. The tokenizer will recognize comment lines.

Parameters:

file - the file containing S-expressions

encoding - the encoding of the file to be tokenized

bufSize - the size of the buffer of the BufferedReader that will be created

Throws:

UnsupportedEncodingException - if the specified encoding is not supported

FileNotFoundException - if the specified file does not exist

IOException

SexpTokenizer

public SexpTokenizer(File file,
                     String encoding,
                     int bufSize,
                     boolean comments)
              throws UnsupportedEncodingException,
                     FileNotFoundException,
                     IOException

Convenience constructor, creating a SexpTokenizer around a BufferedReader around a FileInputStream.

Parameters:

file - the file containing S-expressions

encoding - the encoding of the file to be tokenized

bufSize - the size of the buffer of the BufferedReader that will be created

comments - whether this tokenizer will recognize comments

Throws:

UnsupportedEncodingException - if the specified encoding is not supported

FileNotFoundException - if the specified file does not exist

IOException

SexpTokenizer

public SexpTokenizer(InputStream stream,
                     String encoding,
                     int bufSize)
              throws UnsupportedEncodingException

Convenience constructor, creating a SexpTokenizer around a BufferedReader around a InputStreamReader. The tokenizer will recognize comment lines.

Parameters:

stream - the stream of bytes, encoded with encoding, containing S-expressions

encoding - the encoding of the file to be tokenized

bufSize - the size of the buffer of the BufferedReader that will be created

Throws:

UnsupportedEncodingException - if the specified encoding is not supported

SexpTokenizer

public SexpTokenizer(InputStream stream,
                     String encoding,
                     int bufSize,
                     boolean comments)
              throws UnsupportedEncodingException

Convenience constructor, creating a SexpTokenizer around a BufferedReader around a InputStreamReader.

Parameters:

stream - the stream of bytes, encoded with encoding, containing S-expressions

encoding - the encoding of the file to be tokenized

bufSize - the size of the buffer of the BufferedReader that will be created

comments - whether this tokenizer will recognize comments

Throws:

UnsupportedEncodingException - if the specified encoding is not supported

SexpTokenizer

public SexpTokenizer(URL url,
                     String encoding,
                     int bufSize)
              throws UnsupportedEncodingException,
                     IOException

Convenience constructor, creating a SexpTokenizer around a BufferedReader around an InputStreamReader around the stream created by calling url.openStream(). The tokenizer will recognize comment lines.

Parameters:

url - the url from which to get the stream containing S-expressions

encoding - the encoding of the file to be tokenized

bufSize - the size of the buffer of the BufferedReader

Throws:

UnsupportedEncodingException - if the specified encoding is not supported

IOException - if the underlying call to URL.openStream() or the construction of any of the readers throws an IOException

SexpTokenizer

public SexpTokenizer(URL url,
                     String encoding,
                     int bufSize,
                     boolean comments)
              throws UnsupportedEncodingException,
                     IOException

Convenience constructor, creating a SexpTokenizer around a BufferedReader around a InputStreamReader around the stream created by calling url.openStream().

Parameters:

url - the url from which to get the stream containing S-expressions

encoding - the encoding of the file to be tokenized

bufSize - the size of the buffer of the BufferedReader

comments - whether this tokenizer will recognize comments

Throws:

UnsupportedEncodingException - if the specified encoding is not supported

IOException - if the underlying call to URL.openStream() or the construction of any of the readers throws an IOException