dillonkearns/elm-pages-v3-beta/4.0.0

NOTE: the latest version is 21.0.0

DataSource

In an elm-pages app, each Route Module can define a value data which is a DataSource that will be resolved before init is called. That means it is also available when the page's HTML is pre-rendered during the build step. You can also access the resolved data in head to use it for the page's SEO meta tags.

A DataSource lets you pull in data from:

  • Local files (DataSource.File)
  • HTTP requests (DataSource.Http)
  • Globs, i.e. listing out local files based on a pattern like content/*.txt (DataSource.Glob)
  • Ports, i.e. getting JSON data from running custom NodeJS, similar to a port in a vanilla Elm app except run at build-time in NodeJS, rather than at run-time in the browser (DataSource.Port)
  • Hardcoded data (DataSource.succeed "Hello!")
  • Or any combination of the above, using DataSource.map2, DataSource.andThen, or other combining/continuing helpers from this module

DataSource's vs. Effect's/Cmd's

DataSource's are always resolved before the page is rendered and sent to the browser. A DataSource is never executed in the Browser. Instead, the resolved data from the DataSource is passed down to the Browser - it has been resolved before any client-side JavaScript ever executes. In the case of a pre-rendered route, this is during the CLI build phase, and for server-rendered routes its DataSource is resolved on the server.

Effect's/Cmd's are never executed on the CLI or server, they are only executed in the Browser. The data from a Route Module's init function is used to render the initial HTML on the server or build step, but the Effect isn't executed and update is never called before the page is hydrated in the Browser. This gives a deterministic mental model of what the first render will look like, and a nicely typed way to define the initial Data you have to render your initial view.

Because elm-pages hydrates into a full Elm single-page app, it does need the data in order to initialize the Elm app. So why not just get the data the old-fashioned way, with elm/http, for example?

A few reasons:

  1. DataSource's allow you to pull in data that you wouldn't normally be able to access from an Elm app, like local files, or listings of files in a folder. Not only that, but the dev server knows to automatically hot reload the data when the files it depends on change, so you can edit the files you used in your DataSource and see the page hot reload as you save!
  2. You can pre-render HTML for your pages, including the SEO meta tags, with all that rich, well-typed Elm data available! That's something you can't accomplish with a vanilla Elm app, and it's one of the main use cases for elm-pages.
  3. Because elm-pages has a build step, you know that your DataSource.Http requests succeeded, your decoders succeeded, your custom DataSource validations succeeded, and everything went smoothly. If something went wrong, you get a build failure and can deal with the issues before the site goes live. That means your users won't see those errors, and as a developer you don't need to handle those error cases in your code! Think of it as "parse, don't validate", but for your entire build. In the case of server-rendered routes, a DataSource failure will render a 500 page, so more care needs to be taken to make sure all common errors are handled properly, but the tradeoff is that you can use DataSource's to pull in highly dynamic data and even render user-specific pages.
  4. For static routes, you don't have to worry about an API being down, or hitting it repeatedly. You can build in data and it will end up as optimized binary-encoded data served up with all the other assets of your site. If your CDN (static site host) is down, then the rest of your site is probably down anyway. If your site host is up, then so is all of your DataSource data. Also, it will be served up extremely quickly without needing to wait for any database queries to be performed, andThen requests to be resolved, etc., because all of that work and waiting was done at build-time!

Mental Model

You can think of a DataSource as a declarative (not imperative) definition of data. It represents where to get the data from, and how to transform it (map, combine with other DataSources, etc.).

How do I actually use a DataSource?

This is very similar to Cmd's in Elm. You don't perform a Cmd just by running that code, as you might in a language like JavaScript. Instead, a Cmd will not do anything unless you pass it to The Elm Architecture to have it perform it for you. You pass a Cmd to The Elm Architecture by returning it in init or update. So actually a Cmd is just data describing a side-effect that the Elm runtime can perform, and how to build a Msg once it's done.

DataSource's are very similar. A DataSource doesn't do anything just by "running" it. Just like a Cmd, it's only data that describes a side-effect to perform. Specifically, it describes a side-effect that the elm-pages runtime can perform. There are a few places where we can pass a DataSource to the elm-pages runtime so it can perform it. Most commonly, you give a field called data in your Route Module's definition. Instead of giving a Msg when the side-effects are complete, the page will render once all of the side-effects have run and all the data is resolved. elm-pages makes the resolved data available your Route Module's init, view, update, and head functions, similar to how a regular Elm app passes Msg's in to update.

Any place in your elm-pages app where the framework lets you pass in a value of type DataSource is a place where you can give elm-pages a DataSource to perform (for example, Site.head where you define global head tags for your site).

Basics

type alias DataSource value =
RawRequest value

A DataSource represents data that will be gathered at build time. Multiple DataSources can be combined together using the mapN functions, very similar to how you can manipulate values with Json Decoders in Elm.

map : (a -> b) -> DataSource a -> DataSource b

Transform a request into an arbitrary value. The same underlying HTTP requests will be performed during the build step, but mapping allows you to change the resulting values by applying functions to the results.

A common use for this is to map your data into your elm-pages view:

import DataSource
import Json.Decode as Decode exposing (Decoder)


view =
    DataSource.Http.get
        (Secrets.succeed "https://api.github.com/repos/dillonkearns/elm-pages")
        (Decode.field "stargazers_count" Decode.int)
        |> DataSource.map
            (\stars ->
                { view =
                    \model viewForPage ->
                        { title = "Current stars: " ++ String.fromInt stars
                        , body = Html.text <| "⭐️ " ++ String.fromInt stars
                        , head = []
                        }
                }
            )
succeed : a -> DataSource a

This is useful for prototyping with some hardcoded data, or for having a view that doesn't have any StaticHttp data.

import DataSource


view :
    List ( PagePath, Metadata )
    ->
        { path : PagePath
        , frontmatter : Metadata
        }
    ->
        StaticHttp.Request
            { view : Model -> View -> { title : String, body : Html Msg }
            , head : List (Head.Tag Pages.PathKey)
            }
view siteMetadata page =
    StaticHttp.succeed
        { view =
            \model viewForPage ->
                mainView model viewForPage
        , head = head page.frontmatter
        }
fail : String -> DataSource a

Stop the StaticHttp chain with the given error message. If you reach a fail in your request, you will get a build error. Or in the dev server, you will see the error message in an overlay in your browser (and in the terminal).

fromResult : Result String value -> DataSource value

Turn an Err into a DataSource failure.

Chaining Requests

andThen : (a -> DataSource b) -> DataSource a -> DataSource b

Build off of the response from a previous DataSource request to build a follow-up request. You can use the data from the previous response to build up the URL, headers, etc. that you send to the subsequent request.

import DataSource
import Json.Decode as Decode exposing (Decoder)


licenseData : DataSource String
licenseData =
    DataSource.Http.get
        (Secrets.succeed "https://api.github.com/repos/dillonkearns/elm-pages")
        (Decode.at [ "license", "url" ] Decode.string)
        |> DataSource.andThen
            (\licenseUrl ->
                DataSource.Http.get (Secrets.succeed licenseUrl) (Decode.field "description" Decode.string)
            )
resolve : DataSource (List (DataSource value)) -> DataSource (List value)

Helper to remove an inner layer of Request wrapping.

combine : List (DataSource value) -> DataSource (List value)

Turn a list of StaticHttp.Requests into a single one.

import DataSource
import Json.Decode as Decode exposing (Decoder)


type alias Pokemon =
    { name : String
    , sprite : String
    }


pokemonDetailRequest : StaticHttp.Request (List Pokemon)
pokemonDetailRequest =
    StaticHttp.get
        (Secrets.succeed "https://pokeapi.co/api/v2/pokemon/?limit=3")
        (Decode.field "results"
            (Decode.list
                (Decode.map2 Tuple.pair
                    (Decode.field "name" Decode.string)
                    (Decode.field "url" Decode.string)
                    |> Decode.map
                        (\( name, url ) ->
                            StaticHttp.get (Secrets.succeed url)
                                (Decode.at
                                    [ "sprites", "front_default" ]
                                    Decode.string
                                    |> Decode.map (Pokemon name)
                                )
                        )
                )
            )
        )
        |> StaticHttp.andThen StaticHttp.combine
andMap : DataSource a -> DataSource (a -> b) -> DataSource b

A helper for combining DataSources in pipelines.

map2 : (a -> b -> c) -> DataSource a -> DataSource b -> DataSource c

Like map, but it takes in two Requests.

view siteMetadata page =
    StaticHttp.map2
        (\elmPagesStars elmMarkdownStars ->
            { view =
                \model viewForPage ->
                    { title = "Repo Stargazers"
                    , body = starsView elmPagesStars elmMarkdownStars
                    }
            , head = head elmPagesStars elmMarkdownStars
            }
        )
        (get
            (Secrets.succeed "https://api.github.com/repos/dillonkearns/elm-pages")
            (Decode.field "stargazers_count" Decode.int)
        )
        (get
            (Secrets.succeed "https://api.github.com/repos/dillonkearns/elm-markdown")
            (Decode.field "stargazers_count" Decode.int)
        )
(value1 -> value2 -> value3 -> valueCombined)
-> DataSource value1
-> DataSource value2
-> DataSource value3
-> DataSource valueCombined
(value1 -> value2 -> value3 -> value4 -> valueCombined)
-> DataSource value1
-> DataSource value2
-> DataSource value3
-> DataSource value4
-> DataSource valueCombined
(value1 -> value2 -> value3 -> value4 -> value5 -> valueCombined)
-> DataSource value1
-> DataSource value2
-> DataSource value3
-> DataSource value4
-> DataSource value5
-> DataSource valueCombined
(value1 -> value2 -> value3 -> value4 -> value5 -> value6 -> valueCombined)
-> DataSource value1
-> DataSource value2
-> DataSource value3
-> DataSource value4
-> DataSource value5
-> DataSource value6
-> DataSource valueCombined
( value1
-> value2
-> value3
-> value4
-> value5
-> value6
-> value7
-> valueCombined
)
-> DataSource value1
-> DataSource value2
-> DataSource value3
-> DataSource value4
-> DataSource value5
-> DataSource value6
-> DataSource value7
-> DataSource valueCombined
( value1
-> value2
-> value3
-> value4
-> value5
-> value6
-> value7
-> value8
-> valueCombined
)
-> DataSource value1
-> DataSource value2
-> DataSource value3
-> DataSource value4
-> DataSource value5
-> DataSource value6
-> DataSource value7
-> DataSource value8
-> DataSource valueCombined
( value1
-> value2
-> value3
-> value4
-> value5
-> value6
-> value7
-> value8
-> value9
-> valueCombined
)
-> DataSource value1
-> DataSource value2
-> DataSource value3
-> DataSource value4
-> DataSource value5
-> DataSource value6
-> DataSource value7
-> DataSource value8
-> DataSource value9
-> DataSource valueCombined