Forms Architecture Design

The Basics

There are 2 new concepts/abstractions that will be introduced with the forms feature. These abstractions help us create a generalized data pipeline from our backend server to any other third-party databases with uniform implementation steps.

In this blog, we're going to introduce 2 new abstractions: Command and Transform

Introducing Commands

Let's start with Commands. Let's think about our existing system for a second. As of July 26, 2022 (the time this post is written), our backend system consists of important business logic packages:

• controller → Handles HTTP routing, etc.
• service → The database layer, any database-related queries/mutations go here.
• client → The external (third-party) api calls/logic.
• model → The data structures

Command is a simple interface that abstracts away the implementation of a chain of processes (hence, the name Command ). Defined as the following

type Command interface {	Execute() error}

This is called the command design pattern. The magic of this interface is like any kind of abstraction, you do not need to know what's happening underneath the hood. All you need to do is handle errors. if there's an error in executing a command, simply return that error in the HTTP request (or any error handler).

What is the problem with the existing package structure?

We do not have a place for complex logic to exist. If we're doing a multi-step process (like what we're doing with forms, transform + sending data), we do not yet have a place for the code to live in.

But we do have a place for complex logic, it's the controller, right? The controllers are supposed to be the business logic too!

Yes, and no. Adding stuff to the controller seems like the simplest and easiest way to implement this. But no, it's going to make unit testing harder to do (since you have to add proper context, mock database calls, etc) and harder to read. The upside of adding this complex logic inside the controller doesn't seem enough to justify the downside.

Introducing Transform

Let's dive into Transform. Like what it says, all this abstraction does, is transform a piece of data structure, into another data structure. This is important it should NEVER be able to do anything else!

Transform functions should never mutate the contents of the data, it should translate the data from one structure into another.

Transform is defined as:

type TransformerFunc[  T1 TransformableFrom,   T2 TransformableTo,   S Schema] func(T1, S) (T2, error)

Please note that we're using go1.18 to enable generics

T1 is going to be transformed into T2 with schema S

What is Schema ?

A schema is just another piece of structure that is required to make transformation from T1 to T2 possible.

It answers specific T2 data type generation. For example, you can see that FormToNotion defined as:

func FormToNotion(f model.FormSubmission, s notionapi.Database) (*notionapi.PageCreateRequest, error) {	tf := formToNotionTransformer{		FormSubmission: f,		Properties:     s.Properties,	}	return tf.Transform()}

formToNotion.go

You see that we pass in notionapi.Database as the schema. notionapi.Database schema answers the question of what data type should the data be transformed into? If you have form submitted with Number and MultiSelection type, what data type are those (Number and MultiSelection ) mapped to?

In Notion's case, all of these are defined by notionapi.Database

What is TransformableFrom and what is TransformableTo ?

TransformableFrom defines what kind of structure a data can be transformed from and TransformableTo defines what it can be transformed into.

Currently, we only have:

type TransformableFrom interface {	model.FormSubmission}type TransformableTo interface {	*notionapi.PageCreateRequest | AirtableRowMock}

transform/types.go

This means you can transform model.FormSubmission into *notionapi.PageCreateRequest or AirtableRowMock