Type and Structs



One of the common questions for CUE is how to generate the matching types in a given language. We will introduce the ideas and complexities with type generation while also showing concrete examples of hof gen -T variations.

Types are central to the languages we program in. hof enables us to specify the shape and rules for our types in CUE and then generate them in one or more languages. The schemas and generators for types are the foundation for many other generators and are important to understand.

Generating Types

CUE is a great language for defining types and their validations, but how do we turn them into the structs, classes, and other language specific representations?

Right now, and generally, the answer is text/template. Right new because CUE does not have this capability. Generally because CUE cannot capture the variety and nuances between languages. What are these complications?

  • OOP vs structural typing, how do you express inheriance or embedding?
  • CUE’s types often look like a Venn Diagram with a languages types
  • Native validation will be faster, will also need to be generated.
  • Casing preferences per language
  • Public, private, and protected visibilty
  • Default values, when and where they are setup

It would be a burden to put this all on CUE developers to figure out and maintain. By using text interpolation, we can generate types without modifying CUE. Note, CUE does intend to support some language targets, but there is no timeline for when this will happen yet or what it will look like.

If we want to have a single-source of truth, we need two things

  1. An abstract representation, DSLs are a natural fit
  2. Mappings to our target languages and technologies

CUE happens to be a good language and model for writing and validating both the represenation and mappings.

Type DSLs

We believe that using a DSL, rather than native CUE expressions, is the better option. There are many things which we cannot express directly in CUE types and constraints, and using attributes requires the tool to understand these. So in order to provide maximal flexibility to experiment without needing to modify cue or hof, we use DSLs. Fortunatedly, CUE makes it easy to create and validate DSLs, it’s just a perspective of CUE values afterall.

Another hard question is “is there a single type schema or DSL to rule them all?” Probably not, though one might be able to capture the majority of cases. As defined, the type DSLs and schemas can be extended or specialized, like any CUE value. This will give the community a way to combine and specialize them as needed.

A Type Schema

With hof, we are building some resuable data model schemas. This subsection will show you a simplified version for demonstration.

  • schema
  • example types used

A Type Schema

// Schema for a Type
#Type: {
	// forms the basis for names in target languages
	Name: string

	// This is a CUE pattern for a struct
	// it makes using it later a little easier
	Fields: [field=string]: #Field & { Name: field }

	// How does this type relate to other types
	Relation: [other=string]: "BelongsTo" | "HasOne" | "HasMany" | "ManyToMany"
}

// Schema for a Field
#Field: {
	// forms the basis for names in target languages
	Name: string
	// Normally we would do a better job
	// of validating the Type contents
	Type: string
}

// This is a CUE pattern for a struct
// it makes using it later a little easier, same as fields
[type=string]: #Type & { Name: type }

Example Types

Let’s use a blogging site as our example.

types.cue

User: {
	Fields: {
		id:       Type: "int"
		admin:    Type: "bool"
		username: Type: "string"
		email:    Type: "string"
	}

	Relation: Post: "HasMany"
}

Post: {
	Fields: {
		title:  Type: "string"
		body:   Type: "string"
		public: Type: "bool"
	}

	Relation: User: "BelongsTo"
}

Run cue eval types.cue schema.cue --out yaml to see it’s final form

types.cue

User:
  Name: User
  Fields:
    id:
      Name: id
      Type: int
    admin:
      Name: admin
      Type: bool
    username:
      Name: username
      Type: string
    email:
      Name: email
      Type: string
  Relation:
    Post: HasMany
Post:
  Name: Post
  Fields:
    title:
      Name: title
      Type: string
    body:
      Name: body
      Type: string
    public:
      Name: public
      Type: bool
  Relation:
    User: BelongsTo

The Templates

Now we have to implement the above schema in our target languages and technologies.

We will run all of the following with hof gen types.cue schema.cue -T ...

Output will be put into the out/ directory.

Go Structs

We can start with a single template and file for all types.

Run hof gen types.cue schema.cue -T types.go

or hof gen ... -T "types.go;out/types.go" to write to file

types.go

package types

{{ range . }}
type {{ .Name }} struct {
	{{ range .Fields }}
	{{ camelT .Name }} {{ .Type }}
	{{ end }}
}
{{ end }}

out/types.go

package types

type Post struct {
	Body string

	Public bool

	Title string
}

type User struct {
	Admin bool

	Email string

	Id int

	Username string
}

We can render with repeated templates, which are processed for each element of an iterable (list or struct fields).

Run hof gen types.cue schema.cue -T "type.go;[]out/{{.Name}}.go"

type.go

package types

type {{ .Name }} struct {
	{{ range .Fields }}
	{{ camelT .Name }} {{ .Type }}
	{{ end }}
}

out/User.go

package types

type User struct {
	Admin bool

	Email string

	Id int

	Username string
}

out/Post.go

package types

type Post struct {
	Body string

	Public bool

	Title string
}

Use partial templates for repetitious template content within a single file. Let’s extract field generation into its own template, where we could make it complex. We won’t here, but an example is struct tags for our Go fields. We can also use template helpers in the output filepath.

Run hof gen types.cue schema.cue -P field.go -T "typeP.go;[]out/{{ lower .Name }}.go"

typeP.go

package types

type {{ .Name }} struct {
	{{ range .Fields }}{{ template "field.go" . }}
	{{ end }}
}

field.go

{{ camelT .Name }} {{ .Type }}

out/user.go

package types

type User struct {
	Admin bool

	Email string

	Id int

	Username string
}

SQL & TypeScript

  • multiple templates
  • non-cue type ID (uuid, etc…)

Protobuf

Show issue with indexing, consistent ordering

2 options

More than types

  1. REST & DB lib stubs (not just types)
  • partials, introduce here, or earlier and expand here

Generator Module

Show how to convert to a generator module


More advanced walkthrough and discussion in…

Briefly mention and link to

  1. Generating types from more vanilla CUE (field: string, rather than DSL)
  2. Generate for a framework
2022 Hofstadter, Inc