[Update: in the original wording of this post, I may have blurred the concepts of 'symbol' and 'token'. For the record, a symbol is either a terminal symbol, which represents an elementary textual unit in a grammar, or a non-terminal symbol, which represents…