Testing

It is highly recommended to write tests for your custom checks.

Rego testing

To help you verify the correctness of your custom checks, OPA gives you a framework that you can use to write tests for your checks. By writing tests for your custom checks you can speed up the development process of new rules and reduce the amount of time it takes to modify rules as requirements evolve.

For more details, see Policy Testing.

Example

package user.dockerfile.ID002

test_add_denied {
    r := deny with input as {"stages": {"alpine:3.13": [
        {"Cmd": "add", "Value": ["/target/resources.tar.gz", "resources.jar"]},
        {"Cmd": "add", "Value": ["/target/app.jar", "app.jar"]},
    ]}}

    count(r) == 1
    r[_] == "Consider using 'COPY /target/app.jar app.jar' command instead of 'ADD /target/app.jar app.jar'"
}

To write tests for custom checks, you can refer to existing tests under [trivy-checks][trivy-checks].

Go testing

Fanal which is a core library of Trivy can be imported as a Go library. You can scan config files in Go and test your custom checks using Go's testing methods, such as table-driven tests. This allows you to use the actual configuration file as input, making it easy to prepare test data and ensure that your custom checks work in practice.

In particular, Dockerfile and HCL need to be converted to structural data as input, which may be different from the expected input format.

Tip

We recommend writing OPA and Go tests both since they have different roles, like unit tests and integration tests.

The following example stores allowed and denied configuration files in a directory. Successes contains the result of successes, and Failures contains the result of failures.

{
    name:  "disallowed ports",
    input: "configs/",
    fields: fields{
        policyPaths: []string{"policy"},
        dataPaths:   []string{"data"},
        namespaces:  []string{"user"},
    },
    want: []types.Misconfiguration{
        {
            FileType: types.Dockerfile,
            FilePath: "Dockerfile.allowed",
            Successes: types.MisconfResults{
                {
                    Namespace: "user.dockerfile.ID002",
                    PolicyMetadata: types.PolicyMetadata{
                        ID:          "ID002",
                        Type:        "Docker Custom Check",
                        Title:       "Disallowed ports exposed",
                        Severity:    "HIGH",
                    },
                },
            },
        },
        {
            FileType: types.Dockerfile,
            FilePath: "Dockerfile.denied",
            Failures: types.MisconfResults{
                {
                    Namespace: "user.dockerfile.ID002",
                    Message:   "Port 23 should not be exposed",
                    PolicyMetadata: types.PolicyMetadata{
                        ID:          "ID002",
                        Type:        "Docker Custom Check",
                        Title:       "Disallowed ports exposed",
                        Severity:    "HIGH",
                    },
                },
            },
        },
    },
},

Dockerfile.allowed has one successful result in Successes, while Dockerfile.denied has one failure result in Failures.