[antlr-interest] [C] code to change Token type, use char* and loose data when buffer destroyed

[Ruslan Zasukhin]

Hi All,

===== TASK ======

In SQL we must be able write
      SELECT 'aaa' 'bbbb'

And this should be same as
      SELECT 'aaabbbb'

I.e. Parser must concatenate literals self.
This was quite easy do in ANTLR 2,
and I already have kill 5-6 hours in ANTLR 3.  :-((((((


I have try many tricks for ANTLR3 itself trying to use its tokens and
ANTLR_STRING class but no luck.

Finally I have give up and have try to use simple code as in v2 using
STD::string as place to accumulate literal.

=================================
character_string_literal
@init{ 
    STD::string st;
}
    :    ( STRING_LITERAL
            { 
                st.append(
                    (const char*) $STRING_LITERAL.text->chars,
                    $STRING_LITERAL.text->len );
            } 
        )+
            -> ^( CONST_STR[ st.c_str() ] )
    ;    
=================================

But this not works, because new Token object stores just pointer

        newToken->textState        = ANTLR3_TEXT_CHARP;
        newToken->tokText.chars = (pANTLR3_UCHAR)text;
 
And as only STD::string dies we get problem.


Jim, how this simple task can be solved in the C TARGET ?

Also I see that for Java code they can contruct dynamic text
And produce token using that text. For example on this page

http://www.antlr.org/wiki/display/ANTLR3/Tree+construction

                            -> ^('+' $p INT[String.valueOf($a.int+$b.int)])


But C target tryies to work only which char*


I guess that ANTLR_STRING setText() can help me,
But I cannot see how I can call that from my

            -> ^( CONST_STR[ st.c_str() ] )

???

Thank you for points ...


-- 
Best regards,

Ruslan Zasukhin
VP Engineering and New Technology
Paradigma Software, Inc

Valentina - Joining Worlds of Information
http://www.paradigmasoft.com

[I feel the need: the need for speed]