![]() ![]() I wanted a scanner that supports Unicode and I ran across Russ Cox's page on VM based regular expression engines. This makes it okay for projects where you plan for it ahead of time, but it's still not ideal. I wind up generating a lexer file just to feed it and then more code files to support it, and it can only render one scanner per namespace. It's very fast, and it's easy enough to use, but it's clunky in that I need to generate several files to hook it in to anything I use, and it irritatingly wants a Stream as the only streaming input source - no TextReader or IEnumerable. I began using Garden Points Lexer from Queensland University of Technology as my lexer of choice for projects where I needed Unicode support - most real world significant projects. DFAs can't really do Unicode, and Rolex is DFA based. My Rolex lexer was very efficient, but it couldn't do Unicode - or rather, it could if you didn't mind waiting a week for the table computation to finish. Where to begin? I'm not even sure how to categorize this, except as yet another lexer generator. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |