HuwCampbell/avro/1.0.1

Avro.Schema

This module defines core Avro Schema types and functions for working with them.

Definition

type Schema
= Null
| Boolean
| Int { logicalType : Maybe String }
| Long { logicalType : Maybe String }
| Float
| Double
| Bytes { logicalType : Maybe String }
| String { logicalType : Maybe String }
| Array { items : Schema }
| Map { values : Schema }
| NamedType TypeName
| Record
{ name : TypeName
, aliases : List TypeName
, doc : Maybe String
, fields : List Field
}
| Enum
{ name : TypeName
, aliases : List TypeName
, doc : Maybe String
, symbols : List String
, default : Maybe String
}
| Union { options : List Schema }
| Fixed
{ name : TypeName
, aliases : List TypeName
, size : Int
, logicalType : Maybe String
}

An Avro Schema

type alias Field =
{ name : String
, aliases : List String
, doc : Maybe String
, order : Maybe SortOrder
, type_ : Schema
, default : Maybe Value
}

The Field of a Record

= Ascending
| Descending
| Ignore

Field Sort ordering

Helpers

typeName : Schema -> TypeName

Get the TypeName for an Avro Schema

For primitive types, this is an unqualified name, but for complex types it may be qualified.

Add documentation to a Schema.

If the Schema does not support documentation (i.e, it's not a Record or Enum) this function has no effect.

withAliases : List TypeName -> Schema -> Schema

Add aliases to a Schema.

If the Schema does not support aliases (i.e, it's not a named type) this function has no effect.

Add a logical type to a Schema.

If the Schema does not support a logical type, (e.g., it's a named type) this function has no effect.

Error handling

= TypeMismatch Schema Schema
| MissingField TypeName String
| FieldMismatch TypeName String SchemaMismatch
| MissingUnion TypeName
| MissingSymbol String
| NamedTypeUnresolved TypeName
| FixedWrongSize TypeName Int Int

Errors which can occur when trying to read Avro with an incompatible Schema.

Display a Schema mismatch error.

Schema Validation

= SchemaNestedUnion
| SchemaIdenticalNamesInUnion TypeName
| SchemaHasInvalidFieldName TypeName String String
| SchemaHasInvalidName TypeName String
| SchemaHasDuplicateEnumValue TypeName String
| SchemaEnumDefaultNotFound TypeName String
| SchemaRedefinedType TypeName

Errors which can occur when a Schema is poorly described.

Display a Schema invalid error.

Validates an Avro schema.

Schema's produced using the Codec module or parsed from JSON are typically valid, but there are some ways to create invalid Schemas which may not be well received by other implementations.

This function checks for the following infelicities:

  • Unions must not directly contain other unions,
  • Unions must not ambiguous (contain more than one schemas with the same name or simple type),
  • Enums must not have duplicate values,
  • Enums with a default values must include the value in their options,
  • Type names and record field names are valid, and
  • Named types are not defined more than once.

Other things which are currently not covered by this function:

  • Default values are of the correct type.

The JSON parser will not parse schemas with invalid names or values of the wrong type.

If working with multiple Codecs, it can be a good idea to include a test in your test suite applying this function to your Codec schemas.

Canonical Form

Turn an Avro schema into its canonical form.

Schema canonical form can be used to determine if two schemas are functionally equivalent for writing Avro values, as it strips documentation and aliases as well as normalises names and rendering.

While canonical form technically refers to a formatted JSON string with no whitespace between terms, this function only transforms a Schema value, and one should compose the following functions if the canonical form string is required.

import Avro
import Avro.Schema as Schema exposing (Schema)
import Json.Encode as Encode


renderCanonical : Schema -> String
renderCanonical s =
    Schema.canonical s
        |> Avro.schemaEncoder
        |> Encode.encode 0