ParCinJ logo

Recipe for Writing a Parser

  1. Introducing terminal and non-terminal symbols.

    Example:

    public class ListParser<P extends ListProcessor>
    {
      private static final TerminalSymbolType OPEN = TerminalSymbolType.delimiter("(");
      private static final TerminalSymbolType CLOSE = TerminalSymbolType.delimiter(")");
      private static final TerminalSymbolType COMMA = TerminalSymbolType.delimiter(",");
      
      private NonTerminalSymbol<P> _list;
      
      public ListParser()
      {
        _list = new NonTerminalSymbol<P>("list");
        NonTerminalSymbol<P> element = new NonTerminalSymbol<P>("element");
        TerminalSymbol<P> atom = createAtom();
        TerminalSymbol<P> open = new TerminalSymbolByText<P>(OPEN);
        TerminalSymbol<P> comma = new TerminalSymbolByText<P>(COMMA);
        TerminalSymbol<P> close = new TerminalSymbolByText<P>(CLOSE);
      }
    }
    
  2. Define the non-terminal symbols in terms of other symbols by using ParCinJ's fluent interface API.

    Example:

      public ListParser()
      {
        ...
        _list.definedBy(open
                       .followedBy(element.zeroOrMoreTimesSeparatedBy(comma))
                       .followedBy(close));
        element.definedBy(atom.or(_list));
      }
    
  3. Define an appropriated instance. It should chunk the character stream into Tokens which correspond to the terminal symbols.

    Example:

      public void parse(Reader reader, P processor)
      {
        SimpleLexer lexer = new SimpleLexer(OPEN, CLOSE, COMMA);
        _list.process(lexer.createTokenIterator(reader), processor);
      }
    
    Now the parser can already be used to check syntax:
      new ListParser<Processor>().parse(reader, new Processor() {});
    
    A is thrown in case of syntax violation.
  4. Introduce an interface extending and define methods of useful at certain parsing stages.

    Example:

    public interface ListProcessor extends Processor
    {
      public void startList();
      
      public void handleAtom(String atom);
      
      public void finishList();
    }
    
  5. Anonymously extend the terminal and non-terminal symbol classes used in step 1 to invoke method of the Processor.

    Example:

    public class ListParser<P extends ListProcessor>
    {
      private static final Pattern ATOM_PATTERN = Pattern.compile("[a-z][a-zA-Z0-9]*");
      ...
      public ListParser()
      {
        _list = declareList();
        NonTerminalSymbol<P> element = new NonTerminalSymbol<P>("element");
        TerminalSymbol<P> atom = createAtom();
        TerminalSymbol<P> open = new TerminalSymbolByText<P>(OPEN);
        TerminalSymbol<P> comma = new TerminalSymbolByText<P>(COMMA);
        TerminalSymbol<P> close = new TerminalSymbolByText<P>(CLOSE);
        
        _list.definedBy(open
                       .followedBy(element.zeroOrMoreTimesSeparatedBy(comma))
                       .followedBy(close));
        element.definedBy(atom.or(_list));
      }
      
      private NonTerminalSymbol<P> declareList()
      {
        return new NonTerminalSymbol<P>("list")
          {
            @Override
            protected void preProcess(P processor)
            {
              processor.startList();
            }
    
            @Override
            protected void postProcess(P processor)
            {
              processor.finishList();
            }
          };
      }
    
      private TerminalSymbol<P> createAtom()
      {
        return new TerminalSymbolByType<P>(NormalToken.NORMAL_TOKEN)
          {
            @Override
            protected void process(Token token, P processor)
            {
              String atom = token.getText();
              if (ATOM_PATTERN.matcher(atom).matches() == false)
              {
                throw new IllegalArgumentException("Not a valid atom.");
              }
              processor.handleAtom(atom);
            }
          };
      }
    
  6. Write an implementation of the Processor interface introduced in step 4 to do something useful like
    • pretty formatting,
    • interpreting,
    • building Java objects,
    • compilation.

    Example:

    public class ListBuilder implements ListProcessor
    {
      private final Stack> _stack = new Stack>();
      private List _list = new ArrayList();
      
      public List getList()
      {
        return (List) _list.get(0);
      }
    
      public void startList()
      {
        _stack.push(_list);
        _list = new ArrayList();
      }
      
      public void handleAtom(String atom)
      {
        _list.add(atom);
      }
    
      public void finishList()
      {
        List listAsElement = _list;
        _list = _stack.pop();
        _list.add(listAsElement);
      }
    }
    
     
    
    For the complete example see src.zip of the distribution or browse
    in the 
    source code repository.