|
|||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object | +--org.argouml.util.MyTokenizer
Class for dividing a String into any number of parts. Each part will be a substring of the original String. The first part will at least contain the first character in the string. All following parts will at least contain the first character in the String not covered by any previous part.
The delim parameter to the constructors is a comma separated list of tokens that should be recognized by the tokenizer. These tokens will be returned by the tokenizer as tokens, and any arbitrary text between them will also be returned as tokens. Since the comma has special meaning in this string, it can be escaped with \ to only mean itself (like in "\\,"). For technical reasons it is not possible for any token in this list to be more than 32 characters long.
In addition to the delim parameter it is also possible to use custom separators that allow any string that can be generated by the limited version of a Turing machine that your computer is, to be used as a delimiter.
There are some custom separators provided that you can use to get things like strings in one token. These cannot be used simultaneously by several tokenizers, ie they are not thread safe.
The tokenizer works in a kind of greedy way. When the first separator token from delim is matched or any CustomSeparator returns true from addChar, then it is satisfied it has found a token and does NOT check if it could have found a longer token. Eg: if you have this delim string "<,<<", then "<<" will never be found.
Example
MyTokenizer tzer = new MyTokenizer("Hello, how are you?", " ,\\,"); while (tzer.hasMoreTokens()) System.out.println("\"" + tzer.nextToken() + "\"");
Which whould yield the following output:
"Hello" "," " " "how" " " "are" " " "you?"
CustomSeparator
Field Summary | |
static CustomSeparator |
DOUBLE_QUOTED_SEPARATOR
A custom separator for quoted strings enclosed in double quotes and using \ as escape character. |
static CustomSeparator |
PAREN_EXPR_SEPARATOR
A custom separator for expressions enclosed in parentheses and matching lparams with rparams. |
static CustomSeparator |
PAREN_EXPR_STRING_SEPARATOR
A custom separator for expressions enclosed in parentheses and matching lparams with rparams. |
static CustomSeparator |
SINGLE_QUOTED_SEPARATOR
A custom separator for quoted strings enclosed in single quotes and using \ as escape character. |
Constructor Summary | |
MyTokenizer(java.lang.String string,
java.lang.String delim)
Constructs a new instance. |
|
MyTokenizer(java.lang.String string,
java.lang.String delim,
java.util.Collection seps)
Constructs a new instance. |
|
MyTokenizer(java.lang.String string,
java.lang.String delim,
CustomSeparator sep)
Constructs a new instance. |
Method Summary | |
int |
getTokenIndex()
Returns the index in the string of the last token returned by nextToken, or zero if no token has been retrived. |
boolean |
hasMoreElements()
This class implements the Enumeration interface. |
boolean |
hasMoreTokens()
Returns true if there are more tokens left. |
java.lang.Object |
nextElement()
This class implements the Enumeration interface. |
java.lang.String |
nextToken()
Retrives the next token. |
void |
putToken(java.lang.String s)
Put a token on the input stream. |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
public static final CustomSeparator SINGLE_QUOTED_SEPARATOR
public static final CustomSeparator DOUBLE_QUOTED_SEPARATOR
public static final CustomSeparator PAREN_EXPR_SEPARATOR
public static final CustomSeparator PAREN_EXPR_STRING_SEPARATOR
Constructor Detail |
public MyTokenizer(java.lang.String string, java.lang.String delim)
string
- The String to be tokenized.delim
- The String of delimiters.public MyTokenizer(java.lang.String string, java.lang.String delim, CustomSeparator sep)
string
- The String to be tokenized.delim
- The String of delimiters.sep
- A custom separator to use.public MyTokenizer(java.lang.String string, java.lang.String delim, java.util.Collection seps)
string
- The String to be tokenized.delim
- The String of delimiters.seps
- Some container with custom separators to use.Method Detail |
public boolean hasMoreTokens()
public java.lang.String nextToken()
public java.lang.Object nextElement()
nextElement
in interface java.util.Enumeration
nextToken
public boolean hasMoreElements()
hasMoreElements
in interface java.util.Enumeration
hasMoreTokens
public int getTokenIndex()
public void putToken(java.lang.String s)
The index returned from getTokenIndex will be the same for the token put as that of the last token that wasn't put.
s
- The token to put.
java.lang.NullPointerException
- if s is null.
|
|||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
ArgoUML © 1996-2003 (20030919) | ArgoUML Project Home | ArgoUML Cookbook |