andre-dietrich/parser-combinators/5.1.1

NOTE: the latest version is 7.1.1

Combine

This library provides facilities for parsing structured text data into concrete Elm values.

API Reference

Core Types

type Parser state res

The Parser type.

At their core, Parsers wrap functions from some state and an InputStream to a tuple representing the new state, the remaining InputStream and a ParseResult res.

type alias InputStream =
{ data : String
, input : String
, position : Int
}

The input stream over which Parsers operate.

  • data is the initial input provided by the user
  • input is the remainder after running a parse
  • position is the starting position of input in data after a parse
type alias ParseLocation =
{ source : String
, line : Int
, column : Int
}

A record representing the current parse location in an InputStream.

  • source the current line of source code
  • line the current line number (starting at 1)
  • column the current column (starting at 1)
type alias ParseContext state res =
( state, InputStream, ParseResult res )

A tuple representing the current parser state, the remaining input stream and the parse result. Don't worry about this type unless you're writing your own primitive parsers.

type alias ParseResult res =
Result (List String) res

Running a Parser results in one of two states:

  • Ok res when the parser has successfully parsed the input
  • Err messages when the parser has failed with a list of error messages.
type alias ParseError state =
( state, InputStream, List String )

A tuple representing a failed parse. It contains the state after running the parser, the remaining input stream and a list of error messages.

type alias ParseOk state res =
( state, InputStream, res )

A tuple representing a successful parse. It contains the state after running the parser, the remaining input stream and the result.

Running Parsers

parse : Parser () res -> String -> Result (ParseError ()) (ParseOk () res)

Parse a string. See runParser if your parser needs to manage some internal state.

import Combine.Num exposing (int)
import String


parseAnInteger : String -> Result String Int
parseAnInteger input =
  case parse int input of
    Ok (_, stream, result) ->
      Ok result


    Err (_, stream, errors) ->
      Err (String.join " or " errors)


parseAnInteger "123"
-- Ok 123


parseAnInteger "abc"
-- Err "expected an integer"
Parser state res
-> state
-> String
-> Result (ParseError state) (ParseOk state res)

Parse a string while maintaining some internal state.

import Combine.Num exposing (int)
import String


type alias Output =
  { count : Int
  , integers : List Int
  }


statefulInt : Parse Int Int
statefulInt =
  -- Parse an int, then increment the state and return
  -- the parsed int. It's important that we try to parse
  -- the int _first_ since modifying the state will
  -- always succeed.
  int |> ignore (modifyState ((+) 1))


ints : Parse Int (List Int)
ints =
  sepBy (string " ") statefulInt


parseIntegers : String -> Result String Output
parseIntegers input =
  case runParser ints 0 input of
    Ok (state, stream, ints) ->
      Ok { count = state, integers = ints }


    Err (state, stream, errors) ->
      Err (String.join " or " errors)


parseIntegers ""
-- Ok { count = 0, integers = [] }


parseIntegers "1 2 3 45"
-- Ok { count = 4, integers = [1, 2, 3, 45] }


parseIntegers "1 a 2"
-- Ok { count = 1, integers = [1] }

Constructing Parsers

primitive : (state -> InputStream -> ParseContext state res) -> Parser state res

Construct a new primitive Parser.

If you find yourself reaching for this function often consider opening a Github issue with the library to have your custom Parsers included in the standard distribution.

app : Parser state res -> state -> InputStream -> ParseContext state res

Unwrap a parser so it can be applied to a state and an input stream. This function is useful if you want to construct your own parsers via primitive. If you're using this outside of the context of primitive then you might be doing something wrong so try asking for help on the mailing list.

Here's how you would implement a greedy version of manyTill using primitive and app:

manyTill : Parser s a -> Parser s x -> Parser s (List a)
manyTill p end =
    let
        accumulate acc state stream =
            case app end state stream of
                ( rstate, rstream, Ok _ ) ->
                    ( rstate, rstream, Ok (List.reverse acc) )


                _ ->
                    case app p state stream of
                        ( rstate, rstream, Ok res ) ->
                            accumulate (res :: acc) rstate rstream


                        ( estate, estream, Err ms ) ->
                            ( estate, estream, Err ms )
    in
    primitive <| accumulate []
lazy : (() -> Parser s a) -> Parser s a

Unfortunately this is not a real lazy function anymore, since this functionality is not accessible anymore by ordinary developers. Use this function only to avoid "bad-recursion" errors or use the following example snippet in your code to circumvent this problem:

recursion x =
    \() -> recursion x

Parsers

fail : String -> Parser s a

Fail without consuming any input.

parse (fail "some error") "hello"
-- Err ["some error"]
succeed : a -> Parser s a

Return a value without consuming any input.

parse (succeed 1) "a"
-- Ok 1
string : String -> Parser s String

Parse an exact string match.

parse (string "hello") "hello world"
-- Ok "hello"


parse (string "hello") "goodbye"
-- Err ["expected \"hello\""]
end : Parser s ()

Fail when the input is not empty.

parse end ""
-- Ok ()


parse end "a"
-- Err ["expected end of input"]
whitespace : Parser s String

Parse zero or more whitespace characters.

parse
    (whitespace
        |> keep (string "hello")
    )
    "hello"
-- Ok "hello"


parse
    (whitespace
        |> keep (string "hello")
    )
    "   hello"
-- Ok "hello"
whitespace1 : Parser s String

Parse one or more whitespace characters.

parse
    (whitespace1
        |> keep (string "hello")
    )
    "hello"
-- Err ["whitespace"]


parse
    (whitespace1
        |> keep (string "hello")
    )
    "   hello"
-- Ok "hello"

Regular Expressions

regex : String -> Parser s String

Parse a Regex match.

Regular expressions must match from the beginning of the input and their subgroups are ignored. A ^ is added implicitly to the beginning of every pattern unless one already exists.

parse (regex "a+") "aaaaab"
-- Ok "aaaaa"


parse (regex "a+") "Aaaaab"
-- Err ["expected input matching Regexp /^a+/"]

Use regexWith for more options on allowing case-insensitive or multiline.

regexSub : String -> Parser s ( String, List (Maybe String) )

Parse a Regex match.

Same as regex, but returns also submatches as the second parameter in the result tuple.

parse (regexSub "(a+)(b+)") "aaaaab"
-- Ok ("aaaaab",[Just "aaaaa",Just "b"])


parse (regexSub "(?:a+)(b+)") "aaaaab"
-- Ok ("aaaaab",[Just "b"])
{ caseInsensitive : Bool
, multiline : Bool
}
-> String
-> Parser s String

Parse a Regex match.

Since, Regex now also has support for more parameters, this option was included into this package. Call regexWith with two additional parameters: caseInsensitive and multiline, which allow you to tweak your expression. The rest is as follows. Regular expressions must match from the beginning of the input and their subgroups are ignored. A ^ is added implicitly to the beginning of every pattern unless one already exists.

parse
    (regexWith
        { caseInsensitive = True, multiline = False }
        "a+"
    )
    "AaaAAaAab"
-- Ok "AaaAAaAa"


parse
    (regexWith
        { caseInsensitive = False, multiline = False }
        "a+"
    )
    "AaaAAaAab"
-- Err ["expected input matching Regexp /^a+/"]
{ caseInsensitive : Bool
, multiline : Bool
}
-> String
-> Parser s ( String, List (Maybe String) )

Parse a Regex match.

Similar to regexWith, but a tuple is returned, with a list of additional submatches. The rest is as follows. Regular expressions must match from the beginning of the input and their subgroups are ignored. A ^ is added implicitly to the beginning of every pattern unless one already exists.

parse
    (regexWithSub
        { caseInsensitive = True, multiline = False }
        "a+"
    )
    "AaaAAaAab"
-- Ok ("aaaAAaAa", [])


parse
    (regexWithSub
        { caseInsensitive = False, multiline = False }
        "a+"
    )
    "AaaAAaAab"
-- Err ["expected input matching Regexp /^a+/"]

Combinators

Transforming Parsers

map : (a -> b) -> Parser s a -> Parser s b

Transform the result of a parser.

let
  parser =
    string "a"
      |> map String.toUpper
in
  parse parser "a"
  -- Ok "A"
onsuccess : a -> Parser s x -> Parser s a

Run a parser and return the value on the right on success.

parse (string "true" |> onsuccess True) "true"
-- Ok True


parse (string "true" |> onsuccess True) "false"
-- Err ["expected \"true\""]
mapError : (List String -> List String) -> Parser s a -> Parser s a

Transform the error of a parser.

let
  parser =
    string "a"
      |> mapError (always ["bad input"])
in
  parse parser b
  -- Err ["bad input"]
onerror : String -> Parser s a -> Parser s a

Variant of mapError that replaces the Parser's error with a List of a single string.

parse (string "a" |> onerror "gimme an 'a'") "b"
-- Err ["gimme an 'a'"]

Chaining Parsers

andThen : (a -> Parser s b) -> Parser s a -> Parser s b

Sequence two parsers, passing the result of the first parser to a function that returns the second parser. The value of the second parser is returned on success.

import Combine.Num exposing (int)


choosy : Parser s String
choosy =
  let
    createParser n =
      if n % 2 == 0 then
        string " is even"
      else
        string " is odd"
  in
    int
      |> andThen createParser


parse choosy "1 is odd"
-- Ok " is odd"


parse choosy "2 is even"
-- Ok " is even"


parse choosy "1 is even"
-- Err ["expected \" is odd\""]
andMap : Parser s a -> Parser s (a -> b) -> Parser s b

Sequence two parsers.

import Combine.Num exposing (int)


plus : Parser s String
plus = string "+"


sum : Parser s Int
sum =
  int
    |> map (+)
    |> andMap (plus |> keep int)


parse sum "1+2"
-- Ok 3
sequence : List (Parser s a) -> Parser s (List a)

Run a list of parsers in sequence, accumulating the results. The main use case for this parser is when you want to combine a list of parsers into a single, top-level, parser. For most use cases, you'll want to use one of the other combinators instead.

parse (sequence [string "a", string "b"]) "ab"
-- Ok ["a", "b"]


parse (sequence [string "a", string "b"]) "ac"
-- Err ["expected \"b\""]

Parser Combinators

keep : Parser s a -> Parser s x -> Parser s a

Join two parsers, ignoring the result of the one on the right.

unsuffix : Parser s String
unsuffix =
  regex "[a-z]"
    |> keep (regex "[!?]")


parse unsuffix "a!"
-- Ok "a"
ignore : Parser s x -> Parser s a -> Parser s a

Join two parsers, ignoring the result of the one on the left.

unprefix : Parser s String
unprefix =
  string ">"
    |> ignore (while ((==) ' '))
    |> ignore (while ((/=) ' '))


parse unprefix "> a"
-- Ok "a"
lookAhead : Parser s a -> Parser s a

Apply a parser without consuming any input on success.

parse (lookAhead (string "a") |> keep (string "a")) "a"
-- Ok "a"


parse (lookAhead (string "a") |> keep (string "b")) "a"
-- Err ["expected \"b\""]


parse (lookAhead (string "a") |> keep (string "b")) "b"
-- Err ["expected \"a\""]
while : (Char -> Bool) -> Parser s String

Consume input while the predicate matches.

parse (while ((/=) ' ')) "test 123"
-- Ok "test"
or : Parser s a -> Parser s a -> Parser s a

Choose between two parsers.

parse (or (string "a") (string "b")) "a"
-- Ok "a"


parse (or (string "a") (string "b")) "b"
-- Ok "b"


parse (or (string "a") (string "b")) "c"
-- Err ["expected \"a\"", "expected \"b\""]
choice : List (Parser s a) -> Parser s a

Choose between a list of parsers.

parse (choice [string "a", string "b"]) "a"
-- Ok "a"


parse (choice [string "a", string "b"]) "b"
-- Ok "b"
optional : a -> Parser s a -> Parser s a

Return a default value when the given parser fails.

letterA : Parser s String
letterA = optional "a" (string "a")


parse letterA "a"
-- Ok "a"


parse letterA "b"
-- Ok "a"
maybe : Parser s a -> Parser s (Maybe a)

Wrap the return value into a Maybe. Returns Nothing on failure.

parse (maybe (string "a")) "a"
-- Ok (Just "a")


parse (maybe (string "a")) "b"
-- Ok Nothing
many : Parser s a -> Parser s (List a)

Apply a parser zero or more times and return a list of the results.

parse (many (string "a")) "aaab"
-- Ok ["a", "a", "a"]


parse (many (string "a")) "bbbb"
-- Ok []


parse (many (string "a")) ""
-- Ok []
many1 : Parser s a -> Parser s (List a)

Parse at least one result.

parse (many1 (string "a")) "a"
-- Ok ["a"]


parse (many1 (string "a")) ""
-- Err ["expected \"a\""]
manyTill : Parser s a -> Parser s end -> Parser s (List a)

Apply the first parser zero or more times until second parser succeeds. On success, the list of the first parser's results is returned.

parse
    (string "<!--"
        |> keep
            (manyTill anyChar (string "-->")
                |> map String.fromList
            )
    )
    "<!--foo bar-->"
-- Ok "foo bar"
many1Till : Parser s a -> Parser s end -> Parser s (List a)

Apply the first parser one or more times until second parser succeeds. On success, the list of the first parser's results is returned.

parse
    (string "<!--"
        |> keep
            (many1Till anyChar (string "-->")
                |> map String.fromList
            )
    )
    "<!--foo bar-->"
-- Ok "foo bar"
sepBy : Parser s x -> Parser s a -> Parser s (List a)

Parser zero or more occurrences of one parser separated by another.

parse (sepBy (string ",") (string "a")) "b"
-- Ok []


parse (sepBy (string ",") (string "a")) "a,a,a"
-- Ok ["a", "a", "a"]


parse (sepBy (string ",") (string "a")) "a,a,b"
-- Ok ["a", "a"]
sepBy1 : Parser s x -> Parser s a -> Parser s (List a)

Parse one or more occurrences of one parser separated by another.

parse (sepBy1 (string ",") (string "a")) ""
-- Err ["expected \"a\""]


parse (sepBy1 (string ",") (string "a")) "a"
-- Ok ["a"]


parse (sepBy1 (string ",") (string "a")) "a,"
-- Ok ["a"]
sepEndBy : Parser s x -> Parser s a -> Parser s (List a)

Parse zero or more occurrences of one parser separated and optionally ended by another.

parse (sepEndBy (string ",") (string "a")) "a,a,a,"
-- Ok ["a", "a", "a"]
sepEndBy1 : Parser s x -> Parser s a -> Parser s (List a)

Parse one or more occurrences of one parser separated and optionally ended by another.

parse (sepEndBy1 (string ",") (string "a")) ""
-- Err ["expected \"a\""]


parse (sepEndBy1 (string ",") (string "a")) "a"
-- Ok ["a"]


parse (sepEndBy1 (string ",") (string "a")) "a,"
-- Ok ["a"]
skipMany : Parser s x -> Parser s ()

Apply a parser and skip its result many times.

parse (skipMany (string "a")) "aaa"
-- Ok ()


parse (skipMany (string "a")) ""
-- Ok ()
skipMany1 : Parser s x -> Parser s ()

Apply a parser and skip its result at least once.

parse (skipMany1 (string "a")) "a"
-- Ok ()


parse (skipMany1 (string "a")) ""
-- Err ["expected \"a\""]
chainl : Parser s (a -> a -> a) -> Parser s a -> Parser s a

Parse one or more occurrences of p separated by op, recursively apply all functions returned by op to the values returned by p. See the examples/Calc.elm file for an example.

let
    addop =
        choice
            [ string "+" |> onsuccess (+)
            , string "-" |> onsuccess (-)
            ]
in
parse (chainl addop int) "1+2+3"
-- Ok 6


parse (chainl addop int) "1+2+3-X"
-- Ok 6
chainr : Parser s (a -> a -> a) -> Parser s a -> Parser s a

Similar to chainl but functions of op are applied in right-associative order to the values of p. See the examples/Python.elm file for a usage example.

let
    addop =
        choice
            [ string "+" |> onsuccess (+)
            , string "-" |> onsuccess (-)
            ]
in


parse (chainr addop int) "1-2-3"  -- 1 - (2 - 3)
-- Ok 2


parse (chainl addop int) "1-2-3"  -- 1 - 2 - 3
-- Ok (-4)
count : Int -> Parser s a -> Parser s (List a)

Parse n occurrences of p.

parse (count 3 (string "a")) "aaa"
-- Ok ["a", "a", "a"]


parse (count 3 (string "a")) "aa"
-- Err ["expected \"a\""]


parse (count 3 (string "a")) "aaaaa"
-- Ok ["a", "a", "a"]
between : Parser s l -> Parser s r -> Parser s a -> Parser s a

Parse something between two other parsers.

The parser

parse
    (between
        (string "(")
        (string ")")
        (string "a")
    )
    "(a)"
-- Ok "a"

is equivalent to the parser

string "("
    |> keep (string "a")
    |> ignore (string ")")
parens : Parser s a -> Parser s a

Parse something between parentheses.

parse (parens (string "hello")) "(hello)"
-- Ok "hello"


parse (parens (string "hello")) "(world)"
-- Err ["expected \"hello\""]
braces : Parser s a -> Parser s a

Parse something between braces {}.

parse (braces (string "hello")) "{hello}"
-- Ok "hello"


parse (braces (string "hello")) "{world}"
-- Err ["expected \"hello\""]
brackets : Parser s a -> Parser s a

Parse something between square brackets [].

parse (brackets (string "hello")) "[hello]"
-- Ok "hello"


parse (brackets (string "hello")) "[world]"
-- Err ["expected \"hello\""]

State Combinators

withState : (s -> Parser s a) -> Parser s a

Get the parser's state and pipe it into a parser.

let
    parser =
        string "a"
            |> keep (withState (\state -> succeed state))
in
parse parser "a"
-- Ok ()
putState : s -> Parser s ()

Replace the parser's state.

let
    parser =
        string "a"
            |> andThen (\_ -> putState 42)
in
parse parser "a"
-- Ok ()
modifyState : (s -> s) -> Parser s ()

Modify the parser's state.

let
    parser =
        string "a"
            |> andThen (\_ -> modifyState ((+) 1))
in
parse parser "a"
-- Ok ()
withLocation : (ParseLocation -> Parser s a) -> Parser s a

Get the current position in the input stream and pipe it into a parser.

let
    parser =
        string "a"
            |> keep (withLocation succeed)
in
parse parser "a"
-- Ok { source = "a", line = 0, column = 1 }
withLine : (Int -> Parser s a) -> Parser s a

Get the current line and pipe it into a parser.

let
    parser =
        string "a\n\n"
            |> keep (withLine succeed)
in
parse parser "a\n\n"
-- Ok 2
withColumn : (Int -> Parser s a) -> Parser s a

Get the current column and pipe it into a parser.

let
    parser =
        string "aaa"
            |> keep (withColumn succeed)
in
parse parser "aaa"
-- Ok 3
withSourceLine : (String -> Parser s a) -> Parser s a

Get the current InputStream and pipe it into a parser, only for debugging purposes ...

let
    parser =
        string "a"
            |> keep (withSourceLine succeed)
in
parse parser "abc"
-- Ok "bc"
modifyInput : (String -> String) -> Parser s ()

Modify the parser's current InputStream input (String).

parse
    (modifyInput String.toUpper
        |> keep (many (string "A"))
    )
    "aaa"
-- Ok ["A","A","A"]
putInput : String -> Parser s ()

Replace the remaining input with a new string.

parse
    (string "a"
        |> ignore (putInput "AAA")
        |> keep (many (string "A"))
    )
    "aaa"
-- Ok ["A","A","A"]
modifyPosition : (Int -> Int) -> Parser s ()

Modify the parser's InputStream position (Int).

let
    parser =
        string "a"
            |> ignore (modifyPosition ((+) 1000))
in
parse parser "a"
-- Ok ((),{ data = "a", input = "", position = 1001 },"a")
putPosition : Int -> Parser s ()

Replace the parser position.

let
    parser =
        string "a"
            |> ignore (putPosition 1000)
in
parse parser "a"
-- Ok ((),{ data = "a", input = "", position = 1000 },"a")

Miscellaneous

currentLocation : InputStream -> ParseLocation

Get the current (line, column) in the input stream.

currentSourceLine : InputStream -> String

Get the current source line in the input stream.

currentLine : InputStream -> Int

Get the current line in the input stream.

currentColumn : InputStream -> Int

Get the current column in the input stream.

currentStream : InputStream -> String

Get the current string stream. That might be useful for applying memorization.