PackagesReroutesAboutlamdera.com

jfmengels/elm-review/2.11.1

NOTE: the latest version is 2.16.6

READMEAboutVersionsSourceModules

Review.Test

Module that helps you test your rules, using elm-test.

import Review.Test
import Test exposing (Test, describe, test)
import The.Rule.You.Want.To.Test exposing (rule)


tests : Test
tests =
    describe "The.Rule.You.Want.To.Test"
        [ test "should not report anything when <condition>" <|
            \() ->
                """module A exposing (..)
a = foo n"""
                    |> Review.Test.run rule
                    |> Review.Test.expectNoErrors
        , test "should report Debug.log use" <|
            \() ->
                """module A exposing (..)
a = Debug.log "some" "message" """
                    |> Review.Test.run rule
                    |> Review.Test.expectErrors
                        [ Review.Test.error
                            { message = "Remove the use of `Debug` before shipping to production"
                            , details = [ "Details about the error" ]
                            , under = "Debug.log"
                            }
                        ]
        ]

Strategies for effective testing

Use Test-Driven Development

Writing a rule is a process that works really well with the Test-Driven Development process loop, which is:

  • Before writing any code, write a failing test.
  • Run the test and make sure that it is failing, otherwise you can't be sure that the test is well-written.
  • Write the simplest (almost stupid) code to make the test pass
  • Run the tests again and make sure that the test is passing, and that you didn't break any previous tests
  • Optionally, refactor your code but be sure not to change the behavior of the implementation. You should not add support for new patterns, as you will want to write tests for those first.

Then repeat for every pattern you wish to handle.

Have a good title

A good test title explains

  • what is tested - Probably the rule, but making it explicit in a describe might improve your test report. Or maybe you are testing a sub-part of the rule, and you can name it explicitly.
  • what should happen: (not) reporting an error, fix <something> by <doing something>, ...
  • when: what is the situation that this test sets up?

Ideally, by only reading through the test titles, someone else should be able to rewrite the rule you are testing.

What should you test?

You should test the scenarios where you expect the rule to report something. At the same time, you should also test when it shouldn't. I encourage writing tests to make sure that things that are similar to what you want to report are not reported.

For instance, if you wish to report uses of variables named foo, write a test that ensures that the use of variables named differently does not get reported.

Tests are pretty cheap, and in the case of rules, it is probably better to have too many tests rather than too few, since the behavior of a rule rarely changes drastically.

Design goals

If you are interested, you can read the design goals for this module.

Running tests

The result of running a rule on a String containing source code.

run : Rule -> String -> ReviewResult

Run a Rule on a String containing source code. You can then use expectNoErrors or expectErrors to assert the errors reported by the rule.

import My.Rule exposing (rule)
import Review.Test
import Test exposing (Test, test)


someTest : Test
someTest =
    test "test title" <|
        \() ->
            """
module SomeModule exposing (a)
a = 1"""
                |> Review.Test.run rule
                |> Review.Test.expectNoErrors

The source code needs to be syntactically valid Elm code. If the code can't be parsed, the test will fail regardless of the expectations you set on it.

Note that to be syntactically valid, you need at least a module declaration at the top of the file (like module A exposing (..)) and one declaration (like a = 1). You can't just have an expression like 1 + 2.

Note: This is a simpler version of runWithProjectData. If your rule is interested in project related details, then you should use runWithProjectData instead.

runWithProjectData : Project -> Rule -> String -> ReviewResult

Run a Rule on a String containing source code, with data about the project loaded, such as the contents of elm.json file.

import My.Rule exposing (rule)
import Review.Project as Project exposing (Project)
import Review.Test
import Test exposing (Test, test)


someTest : Test
someTest =
    test "test title" <|
        \() ->
            let
                project : Project
                project =
                    Project.new
                        |> Project.addElmJson elmJsonToConstructManually
            in
            """module SomeModule exposing (a)
a = 1"""
                |> Review.Test.runWithProjectData project rule
                |> Review.Test.expectNoErrors

The source code needs to be syntactically valid Elm code. If the code can't be parsed, the test will fail regardless of the expectations you set on it.

Note that to be syntactically valid, you need at least a module declaration at the top of the file (like module A exposing (..)) and one declaration (like a = 1). You can't just have an expression like 1 + 2.

Note: This is a more complex version of run. If your rule is not interested in project related details, then you should use run instead.

runOnModules : Rule -> List String -> ReviewResult

Run a Rule on several modules. You can then use expectNoErrors or expectErrorsForModules to assert the errors reported by the rule.

This is the same as run, but you can pass several modules. This is especially useful to test rules created with Review.Rule.newProjectRuleSchema, that look at several files, and where the context of the project is important.

import My.Rule exposing (rule)
import Review.Test
import Test exposing (Test, test)


someTest : Test
someTest =
    test "test title" <|
        \() ->
            [ """
module A exposing (a)
a = 1""", """
module B exposing (a)
a = 1""" ]
                |> Review.Test.runOnModules rule
                |> Review.Test.expectNoErrors

The source codes need to be syntactically valid Elm code. If the code can't be parsed, the test will fail regardless of the expectations you set on it.

Note that to be syntactically valid, you need at least a module declaration at the top of each file (like module A exposing (..)) and one declaration (like a = 1). You can't just have an expression like 1 + 2.

Note: This is a simpler version of runOnModulesWithProjectData. If your rule is interested in project related details, then you should use runOnModulesWithProjectData instead.

runOnModulesWithProjectData : Project -> Rule -> List String -> ReviewResult

Run a Rule on several modules. You can then use expectNoErrors or expectErrorsForModules to assert the errors reported by the rule.

This is basically the same as run, but you can pass several modules. This is especially useful to test rules created with Review.Rule.newProjectRuleSchema, that look at several modules, and where the context of the project is important.

import My.Rule exposing (rule)
import Review.Test
import Test exposing (Test, test)


someTest : Test
someTest =
    test "test title" <|
        \() ->
            let
                project : Project
                project =
                    Project.new
                        |> Project.addElmJson elmJsonToConstructManually
            in
            [ """
module A exposing (a)
a = 1""", """
module B exposing (a)
a = 1""" ]
                |> Review.Test.runOnModulesWithProjectData project rule
                |> Review.Test.expectNoErrors

The source codes need to be syntactically valid Elm code. If the code can't be parsed, the test will fail regardless of the expectations you set on it.

Note that to be syntactically valid, you need at least a module declaration at the top of each file (like module A exposing (..)) and one declaration (like a = 1). You can't just have an expression like 1 + 2.

Note: This is a more complex version of runOnModules. If your rule is not interested in project related details, then you should use runOnModules instead.

Making assertions

An expectation for an error. Use error to create one.

expectNoErrors : ReviewResult -> Expectation

Assert that the rule reported no errors. Note, this is equivalent to using expectErrors like expectErrors [].

import Review.Test
import Test exposing (Test, describe, test)
import The.Rule.You.Want.To.Test exposing (rule)


tests : Test
tests =
    describe "The.Rule.You.Want.To.Test"
        [ test "should not report anything when <condition>" <|
            \() ->
                """module A exposing (..)
a = foo n"""
                    |> Review.Test.run rule
                    |> Review.Test.expectNoErrors
        ]
expectErrors : List ExpectedError -> ReviewResult -> Expectation

Assert that the rule reported some errors, by specifying which ones.

Assert which errors are reported using error. The test will fail if a different number of errors than expected are reported, or if the message or the location is incorrect.

import Review.Test
import Test exposing (Test, describe, test)
import The.Rule.You.Want.To.Test exposing (rule)


tests : Test
tests =
    describe "The.Rule.You.Want.To.Test"
        [ test "should report Debug.log use" <|
            \() ->
                """module A exposing (..)
a = Debug.log "some" "message"
"""
                    |> Review.Test.run rule
                    |> Review.Test.expectErrors
                        [ Review.Test.error
                            { message = "Remove the use of `Debug` before shipping to production"
                            , details = [ "Details about the error" ]
                            , under = "Debug.log"
                            }
                        ]
        ]
{ message : String
, details : List String
, under : String
}

Create an expectation for an error.

message should be the message you're expecting to be shown to the user.

under is the part of the code where you are expecting the error to be shown to the user. If it helps, imagine under to be the text under which the squiggly lines will appear if the error appeared in an editor.

tests : Test
tests =
    describe "The.Rule.You.Want.To.Test"
        [ test "should report Debug.log use" <|
            \() ->
                """module A exposing (..)
a = Debug.log "some" "message\""""
                    |> Review.Test.run rule
                    |> Review.Test.expectErrors
                        [ Review.Test.error
                            { message = "Remove the use of `Debug` before shipping to production"
                            , details = [ "Details about the error" ]
                            , under = "Debug.log"
                            }
                        ]
        ]

If there are multiple locations where the value of under appears, the test will fail unless you use atExactly to remove any ambiguity of where the error should be used.

{ start :
{ row : Int
, column : Int
}
, end :
{ row : Int
, column : Int
}
}

Precise the exact position where the error should be shown to the user. This is only necessary when the under field is ambiguous.

atExactly takes a record with start and end positions.

tests : Test
tests =
    describe "The.Rule.You.Want.To.Test"
        [ test "should report multiple Debug.log calls" <|
            \() ->
                """module A exposing (..)
a = Debug.log "foo" z
b = Debug.log "foo" z
"""
                    |> Review.Test.run rule
                    |> Review.Test.expectErrors
                        [ Review.Test.error
                            { message = "Remove the use of `Debug` before shipping to production"
                            , details = [ "Details about the error" ]
                            , under = "Debug.log"
                            }
                            |> Review.Test.atExactly { start = { row = 4, column = 5 }, end = { row = 4, column = 14 } }
                        , Review.Test.error
                            { message = "Remove the use of `Debug` before shipping to production"
                            , details = [ "Details about the error" ]
                            , under = "Debug.log"
                            }
                            |> Review.Test.atExactly { start = { row = 5, column = 5 }, end = { row = 5, column = 14 } }
                        ]
        ]

Tip: By default, do not use this function. If the test fails because there is some ambiguity, the test error will give you a recommendation of what to use as a parameter of atExactly, so you do not have to bother writing this hard-to-write argument yourself.

Create an expectation that the error provides an automatic fix, meaning that it used functions like errorWithFix, and an expectation of what the source code should be after the error's fix have been applied.

In the absence of whenFixed, the test will fail if the error provides a fix. In other words, you only need to use this function if the error provides a fix.

tests : Test
tests =
    describe "The.Rule.You.Want.To.Test"
        [ test "should report multiple Debug.log calls" <|
            \() ->
                """module A exposing (..)
a = 1
b = Debug.log "foo" 2
"""
                    |> Review.Test.run rule
                    |> Review.Test.expectErrors
                        [ Review.Test.error
                            { message = "Remove the use of `Debug` before shipping to production"
                            , details = [ "Details about the error" ]
                            , under = "Debug.log"
                            }
                            |> Review.Test.whenFixed """module SomeModule exposing (b)
a = 1
b = 2
"""
                        ]
        ]
expectErrorsForModules : List ( String, List ExpectedError ) -> ReviewResult -> Expectation

Assert that the rule reported some errors, by specifying which ones and the module for which they were reported.

This is the same as expectErrors, but for when you used runOnModules or runOnModulesWithProjectData. to create the test. When using those, the errors you expect need to be associated with a module. If we don't specify this, your tests might pass because you expected the right errors, but they may be reported for the wrong module!

If you expect the rule to report other kinds of errors or extract data, then you should use the Review.Test.expect and moduleErrors functions.

The expected errors are tupled: the first element is the module name (for example: List or My.Module.Name) and the second element is the list of errors you expect to be reported.

Assert which errors are reported using error. The test will fail if a different number of errors than expected are reported, or if the message or the location is incorrect.

import Review.Test
import Test exposing (Test, test)
import The.Rule.You.Want.To.Test exposing (rule)


someTest : Test
someTest =
    test "should report an error when a module uses `Debug.log`" <|
        \() ->
            [ """
module ModuleA exposing (a)
a = 1""", """
module ModuleB exposing (a)
a = Debug.log "log" 1""" ]
                |> Review.Test.runOnModules rule
                |> Review.Test.expectErrorsForModules
                    [ ( "ModuleB"
                      , [ Review.Test.error
                            { message = "Remove the use of `Debug` before shipping to production"
                            , details = [ "Details about the error" ]
                            , under = "Debug.log"
                            }
                        ]
                      )
                    ]

Assert that the rule reported some errors for the elm.json file, by specifying which ones.

If you expect the rule to report other kinds of errors or extract data, then you should use the Review.Test.expect and elmJsonErrors functions.

test "report an error when a module is unused" <|
    \() ->
        let
            project : Project
            project =
                Project.new
                    |> Project.addElmJson elmJsonToConstructManually
        in
        """
module ModuleA exposing (a)
a = 1"""
            |> Review.Test.runWithProjectData project rule
            |> Review.Test.expectErrorsForElmJson
                [ Review.Test.error
                    { message = "Unused dependency `author/package`"
                    , details = [ "Dependency should be removed" ]
                    , under = "author/package"
                    }
                ]

Assert which errors are reported using error. The test will fail if a different number of errors than expected are reported, or if the message or the location is incorrect.

Assert that the rule reported some errors for the README.md file, by specifying which ones.

If you expect the rule to report other kinds of errors or extract data, then you should use the Review.Test.expect and readmeErrors functions.

test "report an error when a module is unused" <|
    \() ->
        let
            project : Project
            project =
                Project.new
                    |> Project.addReadme { path = "README.md", context = "# Project\n..." }
        in
        """
module ModuleA exposing (a)
a = 1"""
            |> Review.Test.runWithProjectData project rule
            |> Review.Test.expectErrorsForReadme
                [ Review.Test.error
                    { message = "Invalid link"
                    , details = [ "README contains an invalid link" ]
                    , under = "htt://example.com"
                    }
                ]

Assert which errors are reported using error. The test will fail if a different number of errors than expected are reported, or if the message or the location is incorrect.

List
{ message : String
, details : List String
}
-> Expectation

Assert that the rule reported some global errors, by specifying which ones.

If you expect the rule to report other kinds of errors or extract data, then you should use the Review.Test.expect and globalErrors functions.

Assert which errors are reported using records with the expected message and details. The test will fail if a different number of errors than expected are reported, or if the message or details is incorrect.

import Review.Test
import Test exposing (Test, test)
import The.Rule.You.Want.To.Test exposing (rule)


someTest : Test
someTest =
    test "should report a global error when the specified module could not be found" <|
        \() ->
            """
module ModuleA exposing (a)
a = 1"""
                |> Review.Test.run (rule "ModuleB")
                |> Review.Test.expectGlobalErrors
                    [ { message = "Could not find module ModuleB"
                      , details =
                            [ "You mentioned the module ModuleB in the configuration of this rule, but it could not be found."
                            , "This likely means you misconfigured the rule or the configuration has become out of date with recent changes in your project."
                            ]
                      }
                    ]
{ message : String
, details : List String
}
-> Rule
-> Expectation

Assert that the rule will report a configuration error.

import Review.Test
import Test exposing (Test, test)
import The.Rule.You.Want.To.Test exposing (rule)


someTest : Test
someTest =
    test "should report a configuration error when argument is empty" <|
        \() ->
            rule ""
                |> Review.Test.expectConfigurationError
                    { message = "Configuration argument should not be empty"
                    , details = [ "Some details" ]
                    }
expectDataExtract : String -> ReviewResult -> Expectation

Expect the rule to produce a specific data extract.

If you expect the rule to also report errors, then you should use the Review.Test.expect and dataExtract functions.

Note: You do not need to match the exact formatting of the JSON object, though the order of fields does need to match.

import Review.Test
import Test exposing (Test, describe, test)
import The.Rule.You.Want.To.Test exposing (rule)


tests : Test
tests =
    test "should extract the list of fields" <|
        \() ->
            """module A exposing (..)
a = 1
b = 2
"""
                |> Review.Test.run rule
                |> Review.Test.expectDataExtract """
{
    "fields": [ "a", "b" ]
}"""

Composite assertions

expect : List ReviewExpectation -> ReviewResult -> Expectation

Expect multiple outputs for tests.

Functions such as expectErrors and expectGlobalErrors work well, but in some situations a rule will report multiple things: module errors, global errors, errors for elm.json or the README, or even extract data.

When you have multiple expectations to make for a module, use this function.

import Review.Test
import Test exposing (Test, describe, test)
import The.Rule.You.Want.To.Test exposing (rule)


tests : Test
tests =
    describe "The.Rule.You.Want.To.Test"
        [ test "should ..." <|
            \() ->
                [ """module A.B exposing (..)
import B
a = 1
b = 2
c = 3
"""
                , """module B exposing (..)
x = 1
y = 2
z = 3
"""
                ]
                    |> Review.Test.runOnModules rule
                    |> Review.Test.expect
                        [ Review.Test.globalErrors [ { message = "message", details = [ "details" ] } ]
                        , Review.Test.moduleErrors "A.B" [ { message = "message", details = [ "details" ] } ]
                        , Review.Test.dataExtract """
{
    "foo": "bar",
    "other": [ 1, 2, 3 ]
}"""
                        ]
        ]

Expectation of something that the rule will report or do.

Check out the functions below to create these, and then pass them to Review.Test.expect.

Assert that the rule reported some errors for modules, by specifying which ones. To be used along with Review.Test.expect.

If you expect only module errors, then you may want to use expectErrorsForModules which is simpler.

test "report an error when a module is unused" <|
    \() ->
        [ """
module ModuleA exposing (a)
a = 1""", """
module ModuleB exposing (a)
a = Debug.log "log" 1""" ]
            |> Review.Test.runOnModules rule
            |> Review.Test.expect
                [ Review.Test.moduleErrors "ModuleB"
                    [ Review.Test.error
                        { message = "Remove the use of `Debug` before shipping to production"
                        , details = [ "Details about the error" ]
                        , under = "Debug.log"
                        }
                    ]
                ]

Assert which errors are reported using error. The test will fail if a different number of errors than expected are reported, or if the message or the location is incorrect.

List
{ message : String
, details : List String
}

Assert that the rule reported some global errors, by specifying which ones. To be used along with Review.Test.expect.

If you expect only global errors, then you may want to use expectGlobalErrors which is simpler.

Assert which errors are reported using records with the expected message and details. The test will fail if a different number of errors than expected are reported, or if the message or details is incorrect.

import Review.Test
import Test exposing (Test, test)
import The.Rule.You.Want.To.Test exposing (rule)


someTest : Test
someTest =
    test "should report a global error when the specified module could not be found" <|
        \() ->
            """
module ModuleA exposing (a)
a = 1"""
                |> Review.Test.run (rule "ModuleB")
                |> Review.Test.expect
                    [ Review.Test.globalErrors
                        [ { message = "Could not find module ModuleB"
                          , details =
                                [ "You mentioned the module ModuleB in the configuration of this rule, but it could not be found."
                                , "This likely means you misconfigured the rule or the configuration has become out of date with recent changes in your project."
                                ]
                          }
                        ]
                    ]

Assert that the rule reported some errors for the elm.json file, by specifying which ones. To be used along with Review.Test.expect.

If you expect only errors for elm.json, then you may want to use expectErrorsForElmJson which is simpler.

test "report an error when a module is unused" <|
    \() ->
        let
            project : Project
            project =
                Project.new
                    |> Project.addElmJson elmJsonToConstructManually
        in
        """
module ModuleA exposing (a)
a = 1"""
            |> Review.Test.runWithProjectData project rule
            |> Review.Test.expect
                [ Review.Test.elmJson
                    [ Review.Test.error
                        { message = "Unused dependency `author/package`"
                        , details = [ "Dependency should be removed" ]
                        , under = "author/package"
                        }
                    ]
                ]

Assert which errors are reported using error. The test will fail if a different number of errors than expected are reported, or if the message or the location is incorrect.

Assert that the rule reported some errors for the README.md file. To be used along with Review.Test.expect.

If you expect only errors for README.md, then you may want to use expectErrorsForReadme which is simpler.

import Review.Test
import Test exposing (Test, describe, test)
import The.Rule.You.Want.To.Test exposing (rule)


tests : Test
tests =
    describe "The.Rule.You.Want.To.Test"
        [ test "should extract even if there are errors" <|
            \() ->
                let
                    project : Project
                    project =
                        Project.new
                            |> Project.addReadme { path = "README.md", context = "# Project\n..." }
                in
                """module ModuleA exposing (a)
a = 1"""
                    |> Review.Test.runWithProjectData project rule
                    |> Review.Test.expect
                        [ Review.Test.readme
                            [ Review.Test.error
                                { message = "Invalid link"
                                , details = [ "README contains an invalid link" ]
                                , under = "htt://example.com"
                                }
                            ]
                        ]
        ]

Expect the rule to produce a specific data extract. To be used along with Review.Test.expect.

If you expect the rule not to report any errors, then you may want to use expectDataExtract which is simpler.

Note: You do not need to match the exact formatting of the JSON object, though the order of fields does need to match.

import Review.Test
import Test exposing (Test, describe, test)
import The.Rule.You.Want.To.Test exposing (rule)


tests : Test
tests =
    describe "The.Rule.You.Want.To.Test"
        [ test "should extract even if there are errors" <|
            \() ->
                [ """module A.B exposing (..)
import B
a = 1
b = 2
c = 3
"""
                , """module B exposing (..)
x = 1
y = 2
z = 3
"""
                ]
                    |> Review.Test.runOnModules rule
                    |> Review.Test.expect
                        [ Review.Test.dataExtract """
{
    "foo": "bar",
    "other": [ 1, 2, 3 ]
}"""
                        ]
        ]

Deprecated

{ local : List ExpectedError
, global :
List
{ message : String
, details : List String
}
}
-> Expectation

@deprecated Use Review.Test.expect instead.

Assert that the rule reported some global errors and local errors, by specifying which ones.

Use this function when you expect both local and global errors for a particular test, and when you are using run or runWithProjectData. When using runOnModules or runOnModulesWithProjectData, use expectGlobalAndModuleErrors instead.

If you only have local or global errors, you should instead use expectErrors or expectGlobalErrors respectively.

This function works in the same way as expectErrors and expectGlobalErrors.

{ global :
List
{ message : String
, details : List String
}
, modules : List ( String, List ExpectedError )
}
-> Expectation

@deprecated Use Review.Test.expect instead.

Assert that the rule reported some errors for modules and global errors, by specifying which ones.

Use this function when you expect both local and global errors for a particular test, and when you are using runOnModules or runOnModulesWithProjectData. When usingrun or runWithProjectData, use expectGlobalAndLocalErrors instead.

If you only have local or global errors, you should instead use expectErrorsForModules or expectGlobalErrors respectively.

This function works in the same way as expectErrorsForModules and expectGlobalErrors.

Modules

About this package serverReroutesreroutes.json
Pages and docs rendering from package.elm-lang.org, © 2012-2020 Evan Czaplicki. © 2026 Lamdera