cpp.tokenize.Token Class Reference

Inheritance diagram for cpp.tokenize.Token:

Public Member Functions
def	__init__ (self, token_type, name, start, end)
def	__str__ (self)
def	__init__ (self, token_type, name, start, end)
def	__str__ (self)
def	__init__ (self, token_type, name, start, end)
def	__str__ (self)

Public Attributes
	token_type
	name
	start
	end
	whence

Static Private Attributes
	__repr__ = __str__

Detailed Description

Data container to represent a C++ token.

Tokens can be identifiers, syntax char(s), constants, or
pre-processor directives.

start contains the index of the first char of the token in the source
end contains the index of the last char of the token in the source

Definition at line 64 of file tokenize.py.

Constructor & Destructor Documentation

def cpp.tokenize.Token.__init__	(		self,
			token_type,
			name,
			start,
			end
	)

Definition at line 74 of file tokenize.py.

Referenced by cpp.tokenize.Token.__init__().

    def __init__(self, token_type, name, start, end):
        self.token_type = token_type
        self.name = name
        self.start = start
        self.end = end
        self.whence = WHENCE_STREAM

def cpp.tokenize.Token.__init__	(		self,
			token_type,
			name,
			start,
			end
	)

Definition at line 74 of file tokenize.py.

References cpp.tokenize.Token.__init__(), cpp.tokenize.Token.end, cpp.ast.Node.end, message.name, cpp.tokenize.Token.name, cpp.ast.Define.name, cpp.ast.Parameter.name, cpp.ast._GenericDeclaration.name, cpp.ast.Type.name, name, url_test.name, cpp.tokenize.Token.start, cpp.ast.Node.start, cpp.tokenize.Token.token_type, and cpp.tokenize.Token.whence.

    def __init__(self, token_type, name, start, end):
        self.token_type = token_type
        self.name = name
        self.start = start
        self.end = end
        self.whence = WHENCE_STREAM

def cpp.tokenize.Token.__init__	(		self,
			token_type,
			name,
			start,
			end
	)

Definition at line 74 of file tokenize.py.

References cpp.tokenize.Token.__init__(), cpp.tokenize.Token.end, cpp.ast.Node.end, message.name, cpp.tokenize.Token.name, cpp.ast.Define.name, cpp.ast.Parameter.name, cpp.ast._GenericDeclaration.name, cpp.ast.Type.name, name, url_test.name, cpp.tokenize.Token.start, cpp.ast.Node.start, cpp.tokenize.Token.token_type, and cpp.tokenize.Token.whence.

    def __init__(self, token_type, name, start, end):
        self.token_type = token_type
        self.name = name
        self.start = start
        self.end = end
        self.whence = WHENCE_STREAM

Member Function Documentation

def cpp.tokenize.Token.__str__

(

self

)

Definition at line 81 of file tokenize.py.

References cpp.tokenize.Token.end, cpp.ast.Node.end, message.name, cpp.tokenize.Token.name, cpp.ast.Define.name, cpp.ast.Parameter.name, cpp.ast._GenericDeclaration.name, cpp.ast.Type.name, name, url_test.name, cpp.tokenize.Token.start, and cpp.ast.Node.start.

Referenced by cpp.tokenize.Token.__str__().

    def __str__(self):
        if not utils.DEBUG:
            return 'Token(%r)' % self.name
        return 'Token(%r, %s, %s)' % (self.name, self.start, self.end)

def cpp.tokenize.Token.__str__

(

self

)

Definition at line 81 of file tokenize.py.

References cpp.tokenize.Token.__str__(), cpp.tokenize._GetChar(), cpp.tokenize._GetString(), cpp.tokenize.Token.end, cpp.ast.Node.end, cpp.tokenize.GetTokens(), cpp.tokenize.main(), min, message.name, cpp.tokenize.Token.name, cpp.ast.Define.name, cpp.ast.Parameter.name, cpp.ast._GenericDeclaration.name, cpp.ast.Type.name, name, url_test.name, cpp.tokenize.Token.start, and cpp.ast.Node.start.

    def __str__(self):
        if not utils.DEBUG:
            return 'Token(%r)' % self.name
        return 'Token(%r, %s, %s)' % (self.name, self.start, self.end)

def cpp.tokenize.Token.__str__

(

self

)

Definition at line 81 of file tokenize.py.

References cpp.tokenize.Token.__str__(), cpp.tokenize._GetChar(), cpp.tokenize._GetString(), cpp.tokenize.Token.end, cpp.ast.Node.end, cpp.tokenize.GetTokens(), cpp.tokenize.main(), min, message.name, cpp.tokenize.Token.name, cpp.ast.Define.name, cpp.ast.Parameter.name, cpp.ast._GenericDeclaration.name, cpp.ast.Type.name, name, url_test.name, cpp.tokenize.Token.start, and cpp.ast.Node.start.

    def __str__(self):
        if not utils.DEBUG:
            return 'Token(%r)' % self.name
        return 'Token(%r, %s, %s)' % (self.name, self.start, self.end)

Member Data Documentation

cpp.tokenize.Token.__repr__ = __str__

staticprivate

Definition at line 86 of file tokenize.py.

cpp.tokenize.Token.end

Definition at line 78 of file tokenize.py.

Referenced by cpp.tokenize.Token.__init__(), pump.Token.__init__(), and cpp.tokenize.Token.__str__().

cpp.tokenize.Token.name

Definition at line 76 of file tokenize.py.

Referenced by cpp.tokenize.Token.__init__(), and cpp.tokenize.Token.__str__().

cpp.tokenize.Token.start

Definition at line 77 of file tokenize.py.

Referenced by cpp.tokenize.Token.__init__(), pump.Token.__init__(), cpp.tokenize.Token.__str__(), and pump.Token.__str__().

cpp.tokenize.Token.token_type

Definition at line 75 of file tokenize.py.

Referenced by cpp.tokenize.Token.__init__(), pump.Token.__init__(), pump.Token.__str__(), and pump.Token.Clone().

cpp.tokenize.Token.whence

Definition at line 79 of file tokenize.py.

Referenced by cpp.tokenize.Token.__init__().

---

The documentation for this class was generated from the following file:

• proxygen/fizz/build_/gtest/src/gtest/googlemock/scripts/generator/cpp/tokenize.py