elm-tools/parser/1.0.2

NOTE: this package is not compatible with Elm 0.19.1

Parser

Parsers

type alias Parser a =
Parser Context Problem a

A parser! If you have a Parser Int, it is a parser that turns strings into integers.

run : Parser a -> String -> Result Error a

Actually run a parser.

run (keyword "true") "true"  == Ok ()
run (keyword "true") "True"  == Err ...
run (keyword "true") "false" == Err ...

Numbers and Keywords

int : Parser Int

Parse integers. It accepts decimal and hexidecimal formats.

-- decimal
run int "1234" == Ok 1234
run int "1.34" == Err ...
run int "1e31" == Err ...
run int "123a" == Err ...
run int "0123" == Err ...


-- hexidecimal
run int "0x001A" == Ok 26
run int "0x001a" == Ok 26
run int "0xBEEF" == Ok 48879
run int "0x12.0" == Err ...
run int "0x12an" == Err ...

Note: If you want a parser for both Int and Float literals, check out Parser.LanguageKit.number. It does not backtrack, so it should be faster and give better error messages than using oneOf and combining int and float yourself.

Note: If you want to enable octal or binary Int literals, check out Parser.LanguageKit.int.

float : Parser Float

Parse floats.

run float "123"       == Ok 123
run float "3.1415"    == Ok 3.1415
run float "0.1234"    == Ok 0.1234
run float ".1234"     == Ok 0.1234
run float "1e-42"     == Ok 1e-42
run float "6.022e23"  == Ok 6.022e23
run float "6.022E23"  == Ok 6.022e23
run float "6.022e+23" == Ok 6.022e23
run float "6.022e"    == Err ..
run float "6.022n"    == Err ..
run float "6.022.31"  == Err ..

Note: If you want a parser for both Int and Float literals, check out Parser.LanguageKit.number. It does not backtrack, so it should be faster and give better error messages than using oneOf and combining int and float yourself.

Note: If you want to disable literals like .123 like Elm, check out Parser.LanguageKit.float.

symbol : String -> Parser ()

Parse symbols like ,, (, and &&.

run (symbol "[") "[" == Ok ()
run (symbol "[") "4" == Err ... (ExpectingSymbol "[") ...
keyword : String -> Parser ()

Parse keywords like let, case, and type.

run (keyword "let") "let" == Ok ()
run (keyword "let") "var" == Err ... (ExpectingKeyword "let") ...
end : Parser ()

Check if you have reached the end of the string you are parsing.

justAnInt : Parser Int
justAnInt =
  succeed identity
    |= int
    |. end


-- run justAnInt "90210" == Ok 90210
-- run justAnInt "1 + 2" == Err ...
-- run int       "1 + 2" == Ok 1

Parsers can succeed without parsing the whole string. Ending your parser with end guarantees that you have successfully parsed the whole string.

Combining Parsers

succeed : a -> Parser a

A parser that succeeds without consuming any text.

run (succeed 90210  ) "mississippi" == Ok 90210
run (succeed 3.141  ) "mississippi" == Ok 3.141
run (succeed ()     ) "mississippi" == Ok ()
run (succeed Nothing) "mississippi" == Ok Nothing

Seems weird, but it is often useful in combination with oneOf or andThen.

fail : String -> Parser a

A parser always fails.

run (fail "bad list") "[1,2,3]" == Err ..

Seems weird, but it is often useful in combination with oneOf or andThen.

map : (a -> b) -> Parser a -> Parser b

Transform the result of a parser. Maybe you have a value that is an integer or null:

nullOrInt : Parser (Maybe Int)
nullOrInt =
  oneOf
    [ map Just int
    , map (\_ -> Nothing) (keyword "null")
    ]


-- run nullOrInt "0"    == Ok (Just 0)
-- run nullOrInt "13"   == Ok (Just 13)
-- run nullOrInt "null" == Ok Nothing
-- run nullOrInt "zero" == Err ...
zeroOrMore : Parser a -> Parser (List a)

Try to use the parser as many times as possible. Say we want to parse NaN a bunch of times:

batman : Parser Int
batman =
  map List.length (zeroOrMore (keyword "NaN"))


-- run batman "whatever"       == Ok 0
-- run batman ""               == Ok 0
-- run batman "NaN"            == Ok 1
-- run batman "NaNNaN"         == Ok 2
-- run batman "NaNNaNNaN"      == Ok 3
-- run batman "NaNNaN batman!" == Ok 2

Note: If you are trying to parse things like [1,2,3] or { x = 3 } check out the list and record functions in the Parser.LanguageKit module.

oneOf : List (Parser a) -> Parser a

Try a bunch of different parsers. If a parser does not commit, we move on and try the next one. If a parser does commit, we give up on any remaining parsers.

The idea is: if you make progress and commit to a parser, you want to get error messages from that path. If you bactrack and keep trying stuff you will get a much less precise error.

So say we are parsing “language terms” that include integers and lists of integers:

term : Parser Expr
term =
  oneOf
    [ listOf int
    , int
    ]


listOf : Parser a -> Parser (List a)
listOf parser =
  succeed identity
    |. symbol "["
    |. spaces
    ...

When we get to oneOf, we first try the listOf int parser. If we see a [ we commit to that parser. That means if something goes wrong, we do not backtrack. Instead the parse fails! If we do not see a [ we move on to the second option and just try the int parser.

(|=) : Parser (a -> b) -> Parser a -> Parser b

Keep a value in a parser pipeline.

Read about parser pipelines here. They are really nice!

(|.) : Parser keep -> Parser ignore -> Parser keep

Ignore a value in a parser pipeline.

Read about parser pipelines here. They are really nice!

map2 : (a -> b -> value) -> Parser a -> Parser b -> Parser value

This function is not used much in practice. It is nicer to use the parser pipeline operators (|.) and (|=) instead.

That said, this function can combine two parsers. Maybe you want to parse some spaces followed by an integer:

spacesThenInt : Parser Int
spacesThenInt =
  map2 (\_ n -> n) spaces int


spaces : Parser ()
spaces =
  ignoreWhile (\char -> char == ' ')

We can also use map2 to define (|.) and (|=) like this:

(|.) : Parser keep -> Parser ignore -> Parser keep
(|.) keepParser ignoreParser =
  map2 (\keep _ -> keep) keepParser ignoreParser


(|=) : Parser (a -> b) -> Parser a -> Parser b
(|=) funcParser argParser =
  map2 (\func arg -> func arg) funcParser argParser
lazy : (() -> Parser a) -> Parser a

Helper to define recursive parsers. Say we want a parser for simple boolean expressions:

true
false
(true || false)
(true || (true || false))

Notice that a boolean expression might contain other boolean expressions. That means we will want to define our parser in terms of itself:

type Boolean
  = MyTrue
  | MyFalse
  | MyOr Boolean Boolean


boolean : Parser Boolean
boolean =
  oneOf
    [ succeed MyTrue
        |. keyword "true"
    , succeed MyFalse
        |. keyword "false"
    , succeed MyOr
        |. symbol "("
        |. spaces
        |= lazy (\_ -> boolean)
        |. spaces
        |. symbol "||"
        |. spaces
        |= lazy (\_ -> boolean)
        |. spaces
        |. symbol ")"
    ]


spaces : Parser ()
spaces =
  ignoreWhile (\char -> char == ' ')

Notice that boolean uses boolean in its definition! In Elm, you can only define a value in terms of itself it is behind a function call. So lazy helps us define these self-referential parsers.

Note: In some cases, it may be more natural or efficient to use andThen to hide a self-reference behind a function.

andThen : (a -> Parser b) -> Parser a -> Parser b

Run a parser and then run another parser!

Delayed Commits

delayedCommit : Parser a -> Parser value -> Parser value

Only commit if Parser a succeeds and Parser value makes some progress.

This is very important for generating high quality error messages! Read more about this here and here.

delayedCommitMap : (a -> b -> value) -> Parser a -> Parser b -> Parser value

Like delayedCommit, but lets you extract values from both parsers. Read more about it here and here.

Ignoring Stuff

mapWithSource : (String -> a -> b) -> Parser a -> Parser b

Run a parser, and when it is done, extract all of the source code it chomped as well.

This is most useful in combination with the ignore functions, like ignore and ignoreWhile. The idea is that you want to allocate as little as possible, so you ignore all the intermediate stuff and only pull out the final string if you need it.

For example, say we want a parser for variables that start lower case, but then have numbers, letters, and underscores:

variable : Parser String
variable =
  mapWithSource always <|
    ignore 1 Char.isLower
      |. ignoreWhile isVarChar


isVarChar : Char -> Bool
isVarChar char =
  Char.isLower char
  || Char.isUpper char
  || Char.isDigit char
  || char == '_'

The trick is that we only allocate one string here. Otherwise you might allocate the first Char then the remaining String and then put them together with String.cons, allocating a second String.

ignore : Int -> (Char -> Bool) -> Parser ()

Ignore exactly N characters. If you want to ignore one or more spaces, you might say:

oneOrMoreSpaces : Parser ()
oneOrMoreSpaces =
  ignore 1 isSpace
    |. ignoreWhile isSpace


isSpace : Char -> Bool
isSpace char =
  char == ' '

Works well in combination with mapWithSource.

ignoreWhile : (Char -> Bool) -> Parser ()

Ignore zero or more characters. A common use is to ignore spaces:

spaces : Parser ()
spaces =
  ignoreWhile isSpace


isSpace : Char -> Bool
isSpace char =
  char == ' '

Note that this parses zero or more characters, so this parser will always succeed. So our spaces parser will succeed even if isSpace does not match anything.

That said, we only commit to a parser if characters have been consumed. So if you see no spaces, you only commit if the next parser commits.

Works well in combination with mapWithSource.

ignoreUntilBefore : String -> Parser ()

Ignore characters until before the given string.

ignoreUntilAfter : String -> Parser ()

Ignore characters until after the given string. So maybe we want to parse Elm-style single-line comments:

elmComment : Parser ()
elmComment =
  symbol "--"
    |. ignoreUntilAfter "\n"

Or maybe you want to parse JS-style multi-line comments:

jsComment : Parser ()
jsComment =
  symbol "/*"
    |. ignoreUntilAfter "*/"

Note: You must take more care when parsing Elm-style multi-line comments. Elm can recognize nested comments, but the jsComment parser cannot.

Errors

type alias Error =
{ row : Int
, col : Int
, source : String
, problem : Problem
, context : List Context
}

Parse errors as data. You can format it however makes the most sense for your application. Maybe that is all text, or maybe it is fancy interactive HTML. Up to you!

You get:

  • The row and col of the error.
  • The full source provided to the run function.
  • The actual problem you ran into.
  • A stack of context that describes where the error is conceptually.

Note: context is a stack. That means inContext adds to the front of this list, not the back. So if you want the Context closest to the error, you want the first element of the context stack.

type Problem
= BadOneOf (List Problem)
| BadIgnore
| BadInt
| BadFloat
| ExpectingEnd
| ExpectingSymbol String
| ExpectingKeyword String
| ExpectingVariable
| Fail String

The particular problem you ran into.

type alias Context =
{ row : Int
, col : Int
, description : String
}

Most parsers only let you know the row and column where the error occurred. But what if you could also say “the error occured while parsing a list” and let folks know what the parser thinks it is doing?!

The error messages would be a lot nicer! That is what Elm compiler does, and it is what Context helps you do in this library! See the inContext docs for a nice example!

About the actual fields:

Say you use inContext in your list parser. And say get an error trying to parse [ 1, 23zm5, 3 ]. In addition to error information about 23zm5, you would have Context with the row and column of the starting [ symbol.

inContext : String -> Parser a -> Parser a

Specify what you are parsing right now. So if you have a parser for lists like [ 1, 2, 3 ] you could say:

list : Parser (List Int)
list =
  inContext "list" <|
    succeed identity
      |. symbol "["
      |. spaces
      |= commaSep int
      |. spaces
      |. symbol "]"


-- spaces : Parser ()
-- commaSep : Parser a -> Parser (List a)

Now you get that extra context information if there is a parse error anywhere in the list. For example, if you have [ 1, 23zm5, 3 ] you could generate an error message like this:

I ran into a problem while parsing this list:


    [ 1, 23zm5, 3 ]
         ^
Looking for a valid integer, like 6 or 90210.

Notice that the error message knows you are parsing a list right now!