Protocol Documentation

Table of Contents

Top

lang.proto

ClassConstraint

Class constraints

A special constraint type denoting the constraints that occur on the rhs of class definitions. Only used to specify super class constraints in a ClassDef.

Not to be confused with Constraint which denote type class rules.

FieldTypeLabelDescription
class_refTyClassRefType class reference.
argsTyVarrepeatedType variables quantified over ClassDef arguments.

ClassDef

Type class definition

LambdaBuffers use type classes to talk about the various 'meanings' or 'semantics' we want to associate with the types in LambdaBuffers schemata.

For instance, most types can have Eq semantics, meaning they can be compared for equality. Other can have Ord semantics, meaning they can be ordered.

Using type classes and instance declarations, much like in Haskell, users can specify the 'meaning' of each type they declare. For example, serialization in LambdaBuffers is just another type class, it's treated the same as any other type class. Concretely, if we wish to provide JSON serialization for LambdaBuffers types, we declare such a type class and provide desired semantic rules:

module Foo

class JSON a

sum Foo a b = Bar a | Baz b

derive JSON (Foo a b)

Note that for each type class introduced, the Codegen machinery must be updated to support said type class. In other words, it doesn't come for free and for each new type class, a Codegen support must be implemented for any InstanceClause declared by the user. Once all the InstanceClauses have an implementation provided, all the Derived implementation come for free.

FieldTypeLabelDescription
class_nameClassNameType class name.
class_argsTyArgrepeatedType class arguments. Class with no arguments is a trivial class. Compiler MAY report an error. TODO(bladyjoker): MultipleClassArgError.
supersClassConstraintrepeatedSuperclass constraints.
documentationstringDocumentation elaborating on the type class.
source_infoSourceInfoSource information.

ClassName

Type class name

FieldTypeLabelDescription
namestringName ::= [A-Z]+[A-Za-z0-9_]*
source_infoSourceInfoSource information.

ConstrName

Sum type constructor name

FieldTypeLabelDescription
namestringName ::= [A-Z]+[A-Za-z0-9_]*
source_infoSourceInfoSource information.

Constraint

Constraint term

FieldTypeLabelDescription
class_refTyClassRefName of the type class.
argsTyrepeatedConstraint arguments. Constraint with no arguments is a trivial constraint. Compiler MAY report an error.
source_infoSourceInfoSource information.

Derive

Derive statement

Derive statements enable user to specify 'semantic' rules for their types much like InstanceClauses do. However, the Codegen will be able to derive an implementation for any such constraint.

module Prelude

class Eq a

sum Maybe a = Just a | Nothing

derive Eq (Maybe a)

The rule installed for the derive statement is:

eq(maybe(A)) :- eq(just(A) | Nothing).

The rule relates the desired Ty term to its (lambda calculus) 'evaluated' form.

> Currently, there's only support for deriving type class rules and implementations for Ty terms of Kind.KIND_REF_TYPE. That means, type classes like Ord and Eq...

FieldTypeLabelDescription
constraintConstraintConstraint to derive.

FieldName

Record type field name

FieldTypeLabelDescription
namestringName ::= [a-z]+[A-Za-z0-9_]*
source_infoSourceInfoSource information.

InstanceClause

Instance clause

Instance clauses enable users to specify ad-hoc 'semantic' rules for their types. Each such instance must be supported explicitly in the Codegen by providing runtime implementations.

This rule form is used when declaring 'opaque' implementations on Opaque types.

module Prelude

class Eq a

opaque Maybe a

instance Eq a => Eq (Maybe a)

The rule installed for the clause is:

eq(maybe(A)) :- eq(A).

The instance clause is verbatim added to the rule set.

FieldTypeLabelDescription
headConstraintHead of the clause that holds only when the body holds. Type variables introduced in the head of the rule become available in the scope of the body of the rule.
constraintsConstraintrepeatedInstance (rule) body, conjunction of constraints.
source_infoSourceInfoSource information.

Kind

Kinds

A type of a type is called a 'kind'. In Lambda Buffers, all type terms, namely TyArg, TyVar, TyRef, TyApp and TyAbs, are either of kind Type or Type -> Type and Type -> Type -> Type etc.

FieldTypeLabelDescription
kind_refKind.KindRef
kind_arrowKind.KindArrow

Kind.KindArrow

A kind arrow.

FieldTypeLabelDescription
leftKind
rightKind

Module

Module

A module encapsulates type, class and instance definitions.

FieldTypeLabelDescription
module_nameModuleNameModule name.
type_defsTyDefrepeatedType definitions. Duplicate type definitions MUST be reported with ProtoParseError.MultipleTyDefError.
class_defsClassDefrepeatedType class definitions. Duplicate class definitions MUST be reported with ProtoParseError.MultipleClassDefError.
instancesInstanceClauserepeatedType class instance clauses.
derivesDeriverepeatedType class derive statements.
importsModuleNamerepeatedImported modules the Compiler consults when searching for type class rules. TODO(bladyjoker): Rename to ruleImports. Duplicate imports MUST be reported with ProtoParseError.MultipleImportError.
source_infoSourceInfoSource information.

ModuleName

Module name

FieldTypeLabelDescription
partsModuleNamePartrepeatedParts of the module name denoting a hierarchical namespace.
source_infoSourceInfoSource information.

ModuleNamePart

Module name part

FieldTypeLabelDescription
namestringName ::= [A-Z]+[A-Za-z0-9_]*
source_infoSourceInfoSource information.

Opaque

Opaque type.

A type that has an Opaque body represents a 'built-in' or a 'primitive' type that's handled by the semantics 'under the hood'. It's called 'opaque' to denote the fact that the Compiler has no knowledge of its structure, and relies that the necessary knowledge is implemented elsewhere. The Codegen modules for any target language have to be able to handle such types specifically and map to existing value level representations and corresponding types.

Codegen modules would have to implement support for such defined types, for example:

  • In Python Set a would map to set() from the standard library,
  • In Haskell Set a would map to containers.Data.Set.Set type.

Every Opaque type has to be considered deliberately for each language environment targeted by Codegen modules.

TODO(bladyjoker): Consider attaching explicit Kind terms to Opaques.

FieldTypeLabelDescription
source_infoSourceInfoSource information.

Product

A product type term.

FieldTypeLabelDescription
fieldsTyrepeatedFields in a products are types.
source_infoSourceInfoSource information.

Record

A record type term.

FieldTypeLabelDescription
fieldsRecord.FieldrepeatedRecord fields.
source_infoSourceInfoSource information.

Record.Field

Field in a record type.

FieldTypeLabelDescription
field_nameFieldNameRecord field name.
field_tyTyField type.

SourceInfo

Frontend Source information

Frontends are advised to include Source information to denote how their Source* content maps to the Compiler Input. It's essential when reporting Compiler* errors back to the Frontend.

FieldTypeLabelDescription
filestringA filename denoting the Source file.
pos_fromSourcePositionStarting position in Source.
pos_toSourcePositionEnd position in Source.

SourcePosition

Position in Source

FieldTypeLabelDescription
columnint32Column index in the Source.
rowint32Row index in the Source.

Sum

A sum type term.

A type defined as a Sum type is just like a Haskell algebraic data type and represents a sum of products.

FieldTypeLabelDescription
constructorsSum.ConstructorrepeatedSum type constructors. Empty constructors means void and means that the type can't be constructed. Compiler MAY report an error. Duplicate constructors MUST be reported with ProtoParseError.MultipleConstructorError.
source_infoSourceInfoSource information.

Sum.Constructor

Constructor of a Sum type is a Product type term.

FieldTypeLabelDescription
constr_nameConstrNameConstructor name.
productProductProduct type term.

Ty

Type term

A type term that occurs in bodies of type definitions (message TyDef):

sum Maybe a = Just a | Nothing

sum Either a b = Left a | Right b

sum SomeType a = Foo a (Maybe a) | Bar (Either (Maybe a) (SomeType a))

or in instance declarations:

instance Eq (Maybe a)

instance Eq (SomeType Int)

instance (Eq (Maybe a), Eq (SomeType a)) Eq (Either (Maybe a) (SomeType a))

Check out examples.

FieldTypeLabelDescription
ty_varTyVarA type variable.
ty_appTyAppA type application.
ty_refTyRefA type reference.

TyAbs

Type abstraction

A type term that introduces type abstractions (ie. type functions). This type term can only be introduced in the context of a [type definition](@ref TyDef).

FieldTypeLabelDescription
ty_argsTyArgrepeatedList of type variables. No type arguments means delay or const ty_body, meaning TyAbs [] ty_body = ty_body. Duplicate type arguments MUST be reported with ProtoParseError.MultipleTyArgError.
ty_bodyTyBodyType body.
source_infoSourceInfoSource information.

TyApp

Type application

A type term that applies a type abstraction to a list of arguments.

FieldTypeLabelDescription
ty_funcTyType function. TODO(bladyjoker): Rename to ty_abs?
ty_argsTyrepeatedArguments to apply. No arguments to apply means force, meaning `TyApp ty_func [] = ty_func``
source_infoSourceInfoSource information.

TyArg

Type arguments

Arguments in type abstractions.

Type arguments and therefore type variables have kinds, the Compiler only accepts Type kinded type arguments ans therefore type variables.

However, to allow for future evolution if ever necessary, we attach the Kind term to type arguments, even though the Compiler will reject any TyArg that's not of kind Type.

Note, this effectively means that lambda Buffers doesn't support higher-kinded types (ie. HKT).

FieldTypeLabelDescription
arg_nameVarNameArgument name corresponds to variable names.
arg_kindKindArgument kind.
source_infoSourceInfoSource information.

TyBody

Type body.

Lambda Buffers type bodies type terms that can only be specified in the TyAbs context. It's a built-in type term that can only occur enclosed within a TyAbs term which introduces TyVars in the scope of the term.

FieldTypeLabelDescription
opaqueOpaque
sumSum
productProduct
recordRecord

TyClassRef

Type class references

It is necessary to know whether a type class is defined locally or in a foreign module when referring to it in a constraint, this allows users (and requires the frontend) to explicitly communicate that information.

FieldTypeLabelDescription
local_class_refTyClassRef.Local
foreign_class_refTyClassRef.Foreign

TyClassRef.Foreign

Foreign class reference.

FieldTypeLabelDescription
class_nameClassNameForeign module class name.
module_nameModuleNameForeign module name.
source_infoSourceInfoSource information.

TyClassRef.Local

Local type reference.

FieldTypeLabelDescription
class_nameClassNameLocal module class name.
source_infoSourceInfoSource information.

TyDef

Type definition

A type definition consists of a type name and its associated type term.

One way to look at it is that a type definition introduces a named 'type abstraction' in the module scope. Concretely, Either can be considered a type lambda of kind Type -> Type -> Type.

In fact, type definitions are the only way to introduce such types.

Once introduced in the module scope, type definitions are referred to using [TyRef](@ref TyRef) term.

FieldTypeLabelDescription
ty_nameTyNameType name.
ty_absTyAbsType term.
source_infoSourceInfoSource information.

TyName

Type name

FieldTypeLabelDescription
namestringName ::= [A-Z]+[A-Za-z0-9_]*
source_infoSourceInfoSource information.

TyRef

Type reference

A type term that denotes a reference to a type available that's declared locally or in foreign modules.

FieldTypeLabelDescription
local_ty_refTyRef.Local
foreign_ty_refTyRef.Foreign

TyRef.Foreign

Foreign type reference.

FieldTypeLabelDescription
ty_nameTyNameForeign module type name.
module_nameModuleNameForeign module name.
source_infoSourceInfoSource information.

TyRef.Local

Local type reference.

FieldTypeLabelDescription
ty_nameTyNameLocal module type name.
source_infoSourceInfoSource information.

TyVar

Type variable

FieldTypeLabelDescription
var_nameVarNameVariable name.

Tys

A list of type terms useful for debugging

FieldTypeLabelDescription
tiesTyrepeated

VarName

Type variable name

FieldTypeLabelDescription
namestringName ::= [a-z]+
source_infoSourceInfoSource information.

Kind.KindRef

A built-in kind.

NameNumberDescription
KIND_REF_UNSPECIFIED0Unspecified kind SHOULD be inferred by the Compiler.
KIND_REF_TYPE1A Type kind (also know as * in Haskell) built-in.

Top

compiler.proto

Error

Compiler Error

FieldTypeLabelDescription
proto_parse_errorsProtoParseErrorrepeatedErrors occurred during proto parsing.
naming_errorsNamingErrorrepeatedErrors occurred during naming checking.
kind_check_errorsKindCheckErrorrepeatedErrors occurred during kind checking.
ty_class_check_errorsTyClassCheckErrorrepeatedErrors occurred during type class checking.
internal_errorsInternalErrorrepeatedErrors internal to the compiler implementation.

Input

Compiler Input

Compiler Input is a fully self contained list of modules, the entire compilation closure needed by the Compiler to perform its task.

FieldTypeLabelDescription
moduleslambdabuffers.ModulerepeatedModules to compile. Duplicate modules MUST be reported with ProtoParseError.MultipleModuleError.

InternalError

Errors internal to the implementation.

FieldTypeLabelDescription
msgstringError message.
source_infolambdabuffers.SourceInfoSource information if meaningful.

KindCheckError

Kind checking errors.

FieldTypeLabelDescription
unbound_ty_ref_errorKindCheckError.UnboundTyRefError
unbound_ty_var_errorKindCheckError.UnboundTyVarError
unification_errorKindCheckError.UnificationError
cyclic_kind_errorKindCheckError.CyclicKindError
inconsistent_type_errorKindCheckError.InconsistentTypeError

KindCheckError.CyclicKindError

A cyclic kind was encountered. Infinite kinds like this are not acceptable, and we do not support them. We could not construct infinite kind in ty_def.

As the implementation currently stands such an error is (most likely) not representable - therefore not reachable. Such an error would usually occur for a term like: λa. a a - in which case the inference would try to unify two kinds of the form: m and m -> n - because m appears in both terms - the cyclic unification error would be thrown.

In the case of LambdaBuffers - such an error is not (currently) achievable as the kind of the variable is given by the context - (i.e. λa : m . a a, where m is a kind) therefore the unification would fail with Unification Error. Nevertheless - future features might require it.

FieldTypeLabelDescription
ty_deflambdabuffers.TyDef
module_namelambdabuffers.ModuleName

KindCheckError.InconsistentTypeError

The actual_kind differs from the expected_kind.

FieldTypeLabelDescription
module_namelambdabuffers.ModuleName
ty_deflambdabuffers.TyDef
actual_kindlambdabuffers.Kind
expected_kindlambdabuffers.Kind

KindCheckError.UnboundTyRefError

Unbound type reference ty_ref detected in term ty_def.

FieldTypeLabelDescription
module_namelambdabuffers.ModuleName
ty_deflambdabuffers.TyDef
ty_reflambdabuffers.TyRef

KindCheckError.UnboundTyVarError

Unbound variable ty_var detected in term ty_def.

FieldTypeLabelDescription
module_namelambdabuffers.ModuleName
ty_deflambdabuffers.TyDef
ty_varlambdabuffers.TyVar

KindCheckError.UnificationError

In ty_def an error has occurred when trying to unify kind ty_kind_lhs with ty_kind_rhs.

FIXME(cstml): Add source of constraint to the error such that user can see where the constraint was generated - therefore where the error precisely is.

FieldTypeLabelDescription
module_namelambdabuffers.ModuleName
ty_deflambdabuffers.TyDef
ty_kind_lhslambdabuffers.Kind
ty_kind_rhslambdabuffers.Kind

NamingError

Naming error message

FieldTypeLabelDescription
module_name_errlambdabuffers.ModuleNamePart
ty_name_errlambdabuffers.TyName
var_name_errlambdabuffers.VarName
constr_name_errlambdabuffers.ConstrName
field_name_errlambdabuffers.FieldName
class_name_errlambdabuffers.ClassName

Output

Output of the Compiler.

FieldTypeLabelDescription
errorError
resultResult

ProtoParseError

All errors that occur because of Google Protocol Buffer's inability to enforce certain invariants. Some of invariance:

  • using Proto map restricts users to string keys which impacts API documentation, which is why repeated fields are used throughout,
  • using Proto 'oneof' means users have to check if such a field is set or report an error otherwise.
FieldTypeLabelDescription
multiple_module_errorProtoParseError.MultipleModuleError
multiple_tydef_errorProtoParseError.MultipleTyDefError
multiple_classdef_errorProtoParseError.MultipleClassDefError
multiple_tyarg_errorProtoParseError.MultipleTyArgError
multiple_constructor_errorProtoParseError.MultipleConstructorError
multiple_field_errorProtoParseError.MultipleFieldError
multiple_import_errorProtoParseError.MultipleImportError
one_of_not_set_errorProtoParseError.OneOfNotSetError
unknown_enum_errorProtoParseError.UnknownEnumError

ProtoParseError.MultipleClassDefError

Multiple ClassDefs with the same ClassName were found in ModuleName.

FieldTypeLabelDescription
module_namelambdabuffers.ModuleNameModule in which the error was found.
class_defslambdabuffers.ClassDefrepeatedConflicting class definitions.

ProtoParseError.MultipleConstructorError

Multiple Sum Constructors with the same ConstrName were found in ModuleName.TyDef.

FieldTypeLabelDescription
module_namelambdabuffers.ModuleNameModule in which the error was found.
ty_deflambdabuffers.TyDefType definition in which the error was found.
constructorslambdabuffers.Sum.ConstructorrepeatedConflicting constructors.

ProtoParseError.MultipleFieldError

Multiple Record Fields with the same FieldName were found in ModuleName.TyDef.

FieldTypeLabelDescription
module_namelambdabuffers.ModuleNameModule in which the error was found.
ty_deflambdabuffers.TyDefType definition in which the error was found.
fieldslambdabuffers.Record.FieldrepeatedConflicting record fields.

ProtoParseError.MultipleImportError

FieldTypeLabelDescription
module_namelambdabuffers.ModuleNameModule in which the error was found.
importslambdabuffers.ModuleNamerepeatedConflicting module imports.

ProtoParseError.MultipleModuleError

Multiple Modules with the same ModuleName were found.

FieldTypeLabelDescription
moduleslambdabuffers.ModulerepeatedConflicting type definitions.

ProtoParseError.MultipleTyArgError

Multiple TyArgs with the same ArgName were found in ModuleName.TyDef.

FieldTypeLabelDescription
module_namelambdabuffers.ModuleNameModule in which the error was found.
ty_deflambdabuffers.TyDefType definition in which the error was found.
ty_argslambdabuffers.TyArgrepeatedConflicting type abstraction arguments.

ProtoParseError.MultipleTyDefError

Multiple TyDefs with the same TyName were found in ModuleName.

FieldTypeLabelDescription
module_namelambdabuffers.ModuleNameModule in which the error was found.
ty_defslambdabuffers.TyDefrepeatedConflicting type definitions.

ProtoParseError.OneOfNotSetError

Proto oneof field is not set.

FieldTypeLabelDescription
message_namestringProto message name in which the oneof field is not set.
field_namestringThe oneof field that is not set.

ProtoParseError.UnknownEnumError

Proto enum field is unknown.

FieldTypeLabelDescription
enum_namestringProto enum name.
got_tagstringThe unknown tag for the enum.

Result

Compiler Result ~ a successful Compilation Output.

TyClassCheckError

Type class checking errors.

TyClassCheckError.DeriveOpaqueError

In module_name it wasn't possible to solve constraint because a sub_constraint has been derived on an Opaque type. Opaque type can only have an InstanceClause declared for them.

FieldTypeLabelDescription
module_namelambdabuffers.ModuleName
constraintlambdabuffers.Constraint
sub_constraintlambdabuffers.Constraint

TyClassCheckError.ImportNotFoundError

Import missing wasn't found in module_name

FieldTypeLabelDescription
module_namelambdabuffers.ModuleName
missinglambdabuffers.ModuleName

TyClassCheckError.MissingRuleError

In module_name while trying to solve constraint it wasn't possible to find a rule (Derive or InstanceClause) for sub_constraint.

FieldTypeLabelDescription
module_namelambdabuffers.ModuleName
constraintlambdabuffers.Constraint
sub_constraintlambdabuffers.Constraint

TyClassCheckError.OverlappingRulesError

In module_name while trying to solve constraint overlaps (Derive or InstanceClause) were found that could be used to solve the sub_constraint.

FieldTypeLabelDescription
module_namelambdabuffers.ModuleName
constraintlambdabuffers.Constraint
sub_constraintlambdabuffers.Constraint
overlapsTyClassCheckError.OverlappingRulesError.QHeadrepeated

TyClassCheckError.OverlappingRulesError.QHead

NOTE(bladyjoker): This should rather be oneof Derive and InstanceClause.

FieldTypeLabelDescription
module_namelambdabuffers.ModuleName
headlambdabuffers.Constraint

TyClassCheckError.SuperclassCycleError

Superclass cycle cycled_class_refs was detected when checking a class definition for class_name in module module_name.

FieldTypeLabelDescription
module_namelambdabuffers.ModuleName
class_namelambdabuffers.ClassName
cycled_class_refslambdabuffers.TyClassRefrepeated

TyClassCheckError.UnboundClassRefError

Unbound class_ref detected in module_name.

FieldTypeLabelDescription
module_namelambdabuffers.ModuleName
class_reflambdabuffers.TyClassRef

Top

codegen.proto

Error

Codegen Error

FieldTypeLabelDescription
internal_errorsInternalErrorrepeated
unsupported_opaque_errorsUnsupportedOpaqueErrorrepeated
unsupported_class_errorsUnsupportedClassErrorrepeated

Input

Codegen Input

Codegen Input is a fully self contained list of modules, that have been checked by the Compiler.

FieldTypeLabelDescription
moduleslambdabuffers.ModulerepeatedModules to codegen.

InternalError

Errors internal to the implementation.

FieldTypeLabelDescription
msgstringError message.
source_infolambdabuffers.SourceInfoSource information if meaningful.

Output

Codegen output.

FieldTypeLabelDescription
errorError
resultResult

Result

Codegen Result ~ a successful Codegen Output.

UnsupportedClassError

Unsupported Class error for a class class_name defined in module module_name.

FieldTypeLabelDescription
module_namelambdabuffers.ModuleName
class_namelambdabuffers.ClassName

UnsupportedOpaqueError

Unsupported Opaque error for a type ty_name defined in module module_name.

FieldTypeLabelDescription
module_namelambdabuffers.ModuleName
ty_namelambdabuffers.TyName

Scalar Value Types

.proto TypeNotesC++JavaPythonGoC#PHPRuby
doubledoubledoublefloatfloat64doublefloatFloat
floatfloatfloatfloatfloat32floatfloatFloat
int32Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead.int32intintint32intintegerBignum or Fixnum (as required)
int64Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead.int64longint/longint64longinteger/stringBignum
uint32Uses variable-length encoding.uint32intint/longuint32uintintegerBignum or Fixnum (as required)
uint64Uses variable-length encoding.uint64longint/longuint64ulonginteger/stringBignum or Fixnum (as required)
sint32Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s.int32intintint32intintegerBignum or Fixnum (as required)
sint64Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s.int64longint/longint64longinteger/stringBignum
fixed32Always four bytes. More efficient than uint32 if values are often greater than 2^28.uint32intintuint32uintintegerBignum or Fixnum (as required)
fixed64Always eight bytes. More efficient than uint64 if values are often greater than 2^56.uint64longint/longuint64ulonginteger/stringBignum
sfixed32Always four bytes.int32intintint32intintegerBignum or Fixnum (as required)
sfixed64Always eight bytes.int64longint/longint64longinteger/stringBignum
boolboolbooleanbooleanboolboolbooleanTrueClass/FalseClass
stringA string must always contain UTF-8 encoded or 7-bit ASCII text.stringStringstr/unicodestringstringstringString (UTF-8)
bytesMay contain any arbitrary sequence of bytes.stringByteStringstr[]byteByteStringstringString (ASCII-8BIT)