Lukas Renggli

The New Pier Parser

Archive

• July 2020 (2)
• June 2012 (1)
• November 2010 (1)
• October 2010 (2)
• September 2010 (1)
• March 2010 (2)
• February 2010 (2)
• January 2010 (1)
• November 2009 (2)
• October 2009 (2)

Tags

• esug (9)
• magritte (9)
• omnibrowser (8)
• pharo (15)
• pier (15)
• seaside (60)
• smalltalk (29)
• squeak (10)
• talk (12)
• tutorial (9)

The latest version of Pier introduces a new Wiki parser: The previous parser was built using SmaCC and it dates back to the very early versions of SmallWiki. The new parser is hand-written. This might look like a big step back, however there were several compelling reasons not to use an EBNF based parser anymore:

• Extensibility: The hand written parser is fully pluggable and can be extended without having to specify a new grammer. Of course this can happen as part of crosscutting extensions.
• Complexity: The EBNF of the Wiki syntax got very complex over time. With every change, new rules had to be defined in the different contexts of the paragraph, table, or preformatted text. Adding formattings was a nightmare. The internals of link definitions were already parsed manually for quite some time.
• Speed: The hand written parser running through all the pages of my web site is about 4 times faster than the old one.
• Tests: The hand written parser is easier to test, as there is no generated code. Of course all the existing parser tests still pass.

The new parser supports simple formattings that are represented as first class nodes in the resulting Wiki AST. Of course all the new and old tags can be arbitrary nested: italic text with a ^superscript part, monospaced text with a bold part, etc. In my next post I will demonstrate how to extend the parser with a new node type.