Type and Structs

One of the common questions for CUE is how to generate the matching types in a given language. We will introduce the ideas and complexities with type generation while also showing concrete examples of hof gen -T variations.

Types are central to the languages we program in. hof enables us to specify the shape and rules for our types in CUE and then generate them in one or more languages. The schemas and generators for types are the foundation for many other generators and are important to understand.

CUE Discussions

More has been written and discussed at the following links. Keep these handy as you read through.

• https://github.com/cue-lang/cue/discussions/1865
• https://github.com/cue-lang/cue/discussions/1889

Generating Types

CUE is a great language for defining types and their validations, but how do we turn them into the structs, classes, and other language specific representations?

Right now, and generally, the answer is text/template. Right new because CUE does not have this capability. Generally because CUE cannot capture the variety and nuances between languages. What are these complications?

• OOP vs structural typing, how do you express inheriance or embedding?
• CUE’s types often look like a Venn Diagram with a languages types
• Native validation will be faster, will also need to be generated.
• Casing preferences per language
• Public, private, and protected visibility
• Default values, when and where they are setup

It would be a burden to put this all on CUE developers to figure out and maintain. By using text interpolation, we can generate types without modifying CUE. Note, CUE does intend to support some language targets, but there is no timeline for when this will happen yet or what it will look like.

If we want to have a single-source of truth, we need two things

1. An abstract representation, DSLs are a natural fit
2. Mappings to our target languages and technologies

CUE happens to be a good language and model for writing and validating both the representation and mappings.

Type DSLs

We believe that using a DSL, rather than native CUE expressions, is the better option. There are many things which we cannot express directly in CUE types and constraints, and using attributes requires the tool to understand these. So in order to provide maximal flexibility to experiment without needing to modify cue or hof, we use DSLs. Fortunatedly, CUE makes it easy to create and validate DSLs, it’s just a perspective of CUE values afterall.

Another hard question is “is there a single type schema or DSL to rule them all?” Probably not, though one might be able to capture the majority of cases. As defined, the type DSLs and schemas can be extended or specialized, like any CUE value. This will give the community a way to combine and specialize them as needed.

A Type Schema

With hof, we are building some reusable data model schemas. This subsection will show you a simplified version for demonstration.

• schema
• example types used

package example

// here we are applying a schema to our input data
// note how the label is the same in both files
Input: #Input

// This is your input schema
#Input: {
	// this is a CUE pattern to apply #Type to every key
	[key=string]: #Type & { 
		// here we are enriching the input, mapping key -> Name
		Name: key
	}
}

// Schema for a Type
#Type: {
	// Name to use in target languages
	Name: string

	// This is a CUE pattern for a struct of structs
	// you can set nested fields based on the key name in [key=string]
	Fields: [field=string]: #Field & { Name: field }

	// Enum of relation types with the key being the Name of the other side
	Relations: [other=string]: "BelongsTo" | "HasOne" | "HasMany" | "ManyToMany"
}

// Schema for a Field
#Field: {
	// Name to use in target languages
	Name: string
	// the type as a string, for flexibility
	Type: string
}

// note, if you remove "Input" from all of the files
// this will have the same effect, but less control at the top-level
// Top-level schema are helpful when you don't control the input data format
// [key=string]: #Type & { Name: key }

Example Types

Let’s use a blogging site as our example.

package example

// This is our input data, written as CUE
// The schema will be applied to validate
// the inputs and enrich the data model

Input: {
	User: {
		Fields: {
			id:       Type: "int"
			admin:    Type: "bool"
			username: Type: "string"
			email:    Type: "string"
		}
		Relations: {
			Profile: "HasOne"
			Post:    "HasMany"
		}
	}

	Profile: {
		Fields: {
			displayName: Type: "string"
			status:      Type: "string"
			about:       Type: "string"
		}
		Relations: User: "BelongsTo"
	}

	Post: {
		Fields: {
			title:  Type: "string"
			body:   Type: "string"
			public: Type: "bool"
		}
		Relations: User: "BelongsTo"
	}
}

Run cue eval types.cue schema.cue --out yaml to see it’s final form

User:
  Name: User
  Fields:
    id:
      Name: id
      Type: int
    admin:
      Name: admin
      Type: bool
    username:
      Name: username
      Type: string
    email:
      Name: email
      Type: string
  Relation:
    Post: HasMany
Post:
  Name: Post
  Fields:
    title:
      Name: title
      Type: string
    body:
      Name: body
      Type: string
    public:
      Name: public
      Type: bool
  Relation:
    User: BelongsTo

The Templates

Now we have to implement the above schema in our target languages and technologies.

We will run all of the following with hof gen types.cue schema.cue -T ...

Output will be put into the out/ directory.

Go Structs

We can start with a single template and file for all types.

Run hof gen types.cue schema.cue -T types.go

or hof gen ... -T "types.go;out/types.go" to write to file

package types

{{ range .Input }}
type {{ .Name }} struct {
	{{ range .Fields -}}
	{{ camelT .Name }} {{ .Type }}
	{{ end }}
}
{{ end }}

$ hof gen data.cue schema.cue -T types.go

package types

type Post struct {
	Body   string
	Public bool
	Title  string
}

type Profile struct {
	About       string
	DisplayName string
	Status      string
}

type User struct {
	Admin    bool
	Email    string
	Id       int
	Username string
}

We can render with repeated templates, which are processed for each element of an iterable (list or struct fields).

Run hof gen types.cue schema.cue -T "type.go;[]out/{{.Name}}.go"

package types

type {{ .Name }} struct {
	{{ range .Fields }}
	{{ camelT .Name }} {{ .Type }}
	{{ end }}
}

package types

type User struct {
	Admin bool

	Email string

	Id int

	Username string
}

package types

type Post struct {
	Body string

	Public bool

	Title string
}

Use partial templates for repetitious template content within a single file. Let’s extract field generation into its own template, where we could make it complex. We won’t here, but an example is struct tags for our Go fields. We can also use template helpers in the output filepath.

Run hof gen types.cue schema.cue -P field.go -T "typeP.go;[]out/{{ lower .Name }}.go"

package types

{{ range .Input }}
// use a template fragment
{{ template "struct" .}}
{{ end }}

// define a template fragment
{{ define "struct" }}
type {{ .Name }} struct {
	{{ range .Fields }}
	// template from -P flag
	{{ template "field.go" . }}
	{{ end }}
}
{{ end }}

{{ camelT .Name }} {{ .Type }}

package types

type User struct {
	Admin bool

	Email string

	Id int

	Username string
}

SQL & TypeScript

• multiple templates
• non-cue type ID (uuid, etc…)

Protobuf

Show issue with indexing, consistent ordering

2 options

More than types

1. REST & DB lib stubs (not just types)

• partials, introduce here, or earlier and expand here

Generator Module

Show how to convert to a generator module

More advanced walkthrough and discussion in…

Briefly mention and link to

1. Generating types from more vanilla CUE (field: string, rather than DSL)
2. Generate for a framework