Workarounds for a bug in Parser.

The Problem

The Elm Parser internally keeps track of the current position in two ways:

• as a row and a column (like a code editor)
• as an offset into the source string.

See the Positions chapter in the Parser documentation for more details.

Normally both kinds of position infos (row and column vs. offset) are in sync with each other. (For a given source string, you can calculate both row and column from the offset and vice versa.)

There's a bug in the Parser code though. The following parsers break this synchronicity: lineComment, multiComment, chompUntil, and chompUntilEndOr. They set...

• row and column after the (closing) token
• the offset before the (closing) token

Here's an example with chompUntil:

import Parser exposing ((|.), (|=), Parser)


testParser : Parser { row : Int, col : Int, offset : Int }
testParser =
    Parser.succeed (\row col offset -> { row = row, col = col, offset = offset })
        |. Parser.chompUntil "token"
        |= Parser.getRow
        |= Parser.getCol
        |= Parser.getOffset


Parser.run testParser "< token >"
--> Ok { row = 1, col = 8, offset = 2 }

The state after the test parser is run:

• row = 1, col = 8 (corresponding to offset = 7) --> after the token
• offset = 2 (corresponding to row = 1, col = 3) --> before the token

Workaround

As a workaround, this package offers xxxBefore and xxxAfter parsers which consistently position both row/column and offset either before or after the (closing) token.

• lineCommentBefore and lineCommentAfter
• multiCommentBefore and multiCommentAfter
• chompUntilBefore and chompUntilAfter
• chompUntilEndOrBefore and chompUntilEndOrAfter

Why are there two different workarounds for each buggy parser?

On the one hand, if you already have working parsers and don't use the row or column information, then you can replace them with the xxxBefore variants and they will continue to work. (There's one exception for the multiComment parser though, see below.)

On the other hand, most often the xxxAfter variants are easier to use, because you don't need to chomp the (closing) token yourself. So if you write new parsers, you'll likely want to use the xxxAfter variants. Plus, if the bug will be fixed, it can be assumed that the fixed parsers will work like the xxxAfter variants. (If you want to know why, you can look at the description of this pull request.)

Guidelines

If you are unsure whether to use the xxxBefore or the xxxAfter parsers or whether to use the workarounds at all, you could follow these guidelines:

• If you already use one or more of the four buggy parsers:
  • If they work the way you want:
    • If it's likely that you'll need to modify your parsers:
      • Switch to the xxxBefore workarounds. Then you don't have to change your existing parsers, but the workarounds maintain an internally consistent state. This makes it easier to combine them with other parsers later. (There's one exception for the multiComment parser though, see below.)
    • If it's unlikely that you'll need to modify your parsers:
      • Don't change anything (never touch a running system).
  • If they don't work the way you want:
    • Switch to the xxxAfter workarounds, because often they are easier to use and because they implement the intended behavior. You'll have to modify the surrounding parsers, but they don't work anyhow.
• If you don't use one or more of the four buggy parsers yet but are planning to do so:
  • Use the xxxAfter workarounds, because often they are easier to use and because they implement the intended behavior.
• If you don't use any buggy parser yet and are not planning to do so:
  • No buggy parser, no need for a workaround 🙂

Exception

There's one exception to the rules above: if the multiComment parser is used with Nestable comments, then it isn't affected from the bug. (For this mode it's implemented differently.)

Therefore this parser should be replaced with the multiCommentAfter workaround to keep the current behavior.

Parsers