Module jEdit

Package org.gjt.sp.jedit.syntax

Class Token

java.lang.Object

org.gjt.sp.jedit.syntax.Token

Direct Known Subclasses:

Chunk

public class Token
extends Object

A linked list of syntax tokens.

Version:

$Id: Token.java 25244 2020-04-15 15:14:49Z kpouer $

Author:

Slava Pestov

Field Summary

Fields

Modifier and Type	Field	Description
static final byte	COMMENT1
static final byte	COMMENT2
static final byte	COMMENT3
static final byte	COMMENT4
static final byte	DIGIT
static final byte	END
static final byte	FUNCTION
byte	id	The id of this token.
static final byte	ID_COUNT
static final byte	INVALID
static final byte	KEYWORD1
static final byte	KEYWORD2
static final byte	KEYWORD3
static final byte	KEYWORD4
static final byte	LABEL
int	length	The length of this token.
static final byte	LITERAL1
static final byte	LITERAL2
static final byte	LITERAL3
static final byte	LITERAL4
static final byte	MARKUP
Token	next	The next token in the linked list.
static final byte	NULL
int	offset	The start offset of this token.
static final byte	OPERATOR
ParserRuleSet	rules	The rule set of this token.
static final String[]	TOKEN_TYPES

Constructor Summary

Constructors

Constructor	Description
Token(byte id, int offset, int length, ParserRuleSet rules)	Creates a new token.

Method Summary

Modifier and Type	Method	Description
static boolean	isCommentOrLiteral(byte id)
static byte	stringToToken(String value)	Converts a token type string to a token type constant.
static String	tokenToString(byte token)	Converts a token type constant to a token type string.
String	toString()	Returns a string representation of this token.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Details

TOKEN_TYPES

public static final String[] TOKEN_TYPES

NULL

public static final byte NULL

See Also:

• Constant Field Values

COMMENT1

public static final byte COMMENT1

See Also:

• Constant Field Values

COMMENT2

public static final byte COMMENT2

See Also:

• Constant Field Values

COMMENT3

public static final byte COMMENT3

See Also:

• Constant Field Values

COMMENT4

public static final byte COMMENT4

See Also:

• Constant Field Values

DIGIT

public static final byte DIGIT

See Also:

• Constant Field Values

FUNCTION

public static final byte FUNCTION

See Also:

• Constant Field Values

INVALID

public static final byte INVALID

See Also:

• Constant Field Values

KEYWORD1

public static final byte KEYWORD1

See Also:

• Constant Field Values

KEYWORD2

public static final byte KEYWORD2

See Also:

• Constant Field Values

KEYWORD3

public static final byte KEYWORD3

See Also:

• Constant Field Values

KEYWORD4

public static final byte KEYWORD4

See Also:

• Constant Field Values

LABEL

public static final byte LABEL

See Also:

• Constant Field Values

LITERAL1

public static final byte LITERAL1

See Also:

• Constant Field Values

LITERAL2

public static final byte LITERAL2

See Also:

• Constant Field Values

LITERAL3

public static final byte LITERAL3

See Also:

• Constant Field Values

LITERAL4

public static final byte LITERAL4

See Also:

• Constant Field Values

MARKUP

public static final byte MARKUP

See Also:

• Constant Field Values

OPERATOR

public static final byte OPERATOR

See Also:

• Constant Field Values

ID_COUNT

public static final byte ID_COUNT

See Also:

• Constant Field Values

END

public static final byte END

See Also:

• Constant Field Values

id

public byte id

The id of this token.

offset

public int offset

The start offset of this token.

length

public int length

The length of this token.

rules

public ParserRuleSet rules

The rule set of this token.

next

public Token next

The next token in the linked list.

Constructor Details

Token

public Token(byte id,
 int offset,
 int length,
 ParserRuleSet rules)

Creates a new token.

Parameters:

id - The id of the token

offset - The start offset of the token

length - The length of the token

rules - The parser rule set that generated this token

Method Details

stringToToken

public static byte stringToToken(String value)

Converts a token type string to a token type constant.

Parameters:

value - The token type

Since:

jEdit 4.1pre1

tokenToString

public static String tokenToString(byte token)

Converts a token type constant to a token type string.

Since:

jEdit 4.2pre1

toString

public String toString()

Returns a string representation of this token.

Overrides:

toString in class Object

isCommentOrLiteral

public static boolean isCommentOrLiteral(byte id)

Returns:

true for COMMENTx and LITERALx tokens