In this post I will try to go through high-level design aspects of domainql-form. How I initially envisioned them to work and how they evolved over time driven by our experiences with it.
First concept
The first concept was really simple. We have a MobX domain object that represents the root object for the part of the domain we manage with the form, and then we have the <Form/> component that receives the object as value prop.
The <Field/> components reference paths within the object graph by lodash like paths. Here for our example “owner.name”, which first references the owner object that is embedded as property within the root object and then the name property within that owner object.
I mostly imagined one big <Form/> component per view.
This works and for large parts still works this way, but over time we slowly evolved into a more complex model.
Problem: HTML
The first thing that became on issue was the of course well-known fact that you can’t nest forms within HTML. And at first it seems like, duh, who would do something like that? But then you get a use-case where you have some form fields and then data-table connected to that object and then some more form fields. And of course, the <Datagrid/> needs to do all kinds of form control related things like choosing pagination sizes, letting the user enter filter values etc pp.
Problem: Awkard forms
While in some cases it is just natural to have sub-ordinate objects connected in the form and to edit fields within those, but as soon as you get to lists or deeply nested property paths it gets awkward very quickly, especially if you have to construct the paths dynamically.
Solution: Many Forms Paradigm
So we obviously need to be able to have many forms, often referencing the same object. But we also have cases where we want to edit the nth element out of a list of associated entities.
The form components now point anywhere they like. One root object, many root objects, objects within root objects, doesn’t matter.
We just have many forms that write into the same (non-isolated) objects. These <Form/> components all have their own <form/> elements. But what we want most of the time is that the forms behave as if they were one form.
If the user has entered erroneous data and there is an error displayed, all non-discarding buttons must be disabled. Only things like “Cancel” can remain enabled.
FormContext
This orchestration of <Form/> component functionality is handled by the new FormContext class. There is a default context that is always used unless the application author created and referenced another FormContext. e.g. For processes that have to independent form-flows side-by-side or a main process and a sidebar-process.
The FormContext also registers all available memoized field-contexts which can be used to implement high-level form behavior on top of domainql-form.
For the second post in our “Cool shit in Automaton” series, I’d like to write about the InteractiveQuery mechanism. From the general conception to the final implementation with some nice Automaton features.
This post might be conceptually interesting for everyone involved with GraphQL, concrete it is most useful if you’re using Automaton.
Design
The InteractiveQuery mechanism was designed to drive the data needs of our widgets. It is a set of high-level abstract query definition types that offer interactive pagination, filtering, sorting and configuring of GraphQL queries.
It touches many Automaton subsystems which I will discuss in some detail here although it is not strictly necessary to understand all of it to use it. In many ways it’s pretty magical.
Defining an InteractiveQuery based GraphQL Query
One of design goals was of course user-friendliness but not at the expense of power and general applicability. A part of that being that we want to write the least amount of code possible, but still retain all the flexibility we need in our client’s domains.
InteractiveQuery turned out to be the driving force behind the Degenerification feature in DomainQL. The basic idea is that we want to be able to use Generics in Java and translate that into the GraphQL type world.
Let’s look at the default GraphQL query implementation for an example entity.
/**
* Default implementation of an InteractiveQuery based query for type [T].
*
* @param type
* @param env
* @param config configuration for the Interactive query.
* @param <T>
* @return
*/@GraphQLQuerypublic <T> InteractiveQuery<T> iQuery(
@GraphQLTypeParam(
types = {
Foo.class
}
)
Class<T> type,
DataFetchingEnvironment env,
@NotNull QueryConfig config
)
{
log.info("iQuery<{}>, config = {}", type, config);
return interactiveQueryService.buildInteractiveQuery( type, env, config)
.execute();
}
Code language:Java(java)
This is the standard implementation for InteractiveQuery based GraphQL queries. The first method parameter “type” controls the generation of types for this generic method. For each concrete type listed, a degenerified variant of the iQuery method will be defined, by default appending the name of the concrete type to the method method name (here “iQueryFoo”). This is not the most pretty solution but it does keep things in order especially with huge domains. Our app schema already contains 42 variants of this and we’ve just started.
InteractiveQuery in GraphQL
Let’s take a look at the GraphQL types generated from that Java code snippet.
type QueryType {
"Default implementation of an InteractiveQuery based query for type Foo." iQueryFoo(config: QueryConfigInput!): InteractiveQueryFoo
}
"Interactive Query with Foo payload."type InteractiveQueryFoo {
"Column states for the current result."columnStates: [ColumnState]
"Query configuration the current result was produced with."queryConfig: QueryConfig
"Total row count available."rowCount: Int
"List with current rows of Foo."rows: [Foo]
"Name of payload type (always 'Foo')"type: String}
"The state of a column within an interactive query."type ColumnState {
"True if column is enabled. Server might disabled columns."enabled: Boolean"Column name"name: String"True if the column is sortable."sortable: Boolean}
"Encapsulates all parameters of an interactive query."type QueryConfig {
"FilterDSL condition graph or null"condition: Condition
"Current page within the paginated results"offset: Int
"Optional unique query identifier. Useful for server-side query implementations."id: String"Maximum number of paginated results.,"pageSize: Int
"Current sort order for the query."sortFields: [FieldExpression]
}
"Map graph representing JOOQ conditions"scalar Condition
"Map graph representing a JOOQ field expression"scalar FieldExpression
Code language:JavaScript(javascript)
The iQueryFoo query returns the InteractiveQueryFoo type which is a degenerification of InteractiveQuery<T> for the type Foo.
The InteractiveQuery types are all structurally the same and only differ in the payload type. The “rows” field contains a List of the payload type.
Filtering and Sorting
Here is where the real magic starts. So we want to filter a query, but we don’t want to write code for it. The widget provides a JSON description of a filter which we then apply in the context of the current SQL query.
FilterDSL
There is another complication in that while we / Automaton clearly prefers PostgreSQL as database, our clients may or may not, so we will support all databases that JOOQ supports.
Which also means that we wanted to avoid mucking around with SQL in that degree in any case, i.e. we needed an actual abstraction.
Luckily, JOOQ comes with it’s own Condition API which we can just adapt.
So we created the FilterDSL which is a pretty close copy of the JOOQ condition DSL with some necessary additions.
./src/main/js/apps/myapp/queries/Q_Foo.js
import { query } from"@quinscape/automaton-js"exportdefault query(
// language=GraphQL`query iQueryFoo($config: QueryConfigInput!)
{
iQueryFoo(config: $config)
{
type
columnStates{
name
enabled
sortable
}
queryConfig{
id
condition
currentPage
pageSize
sortFields
}
rows{
id
name
description
flag
type
owner{
id
login
}
}
rowCount
}
}`,
{
"config": {
"pageSize": 20
}
}
)
Code language:JavaScript(javascript)
Here we see the query definition to query our Foo type. It has fields of different scalar types and an embedded owner object with an id and a login name.
or a mix in between. Personally I find the second style not so good and the dangling .not() is outright horrible.
Using value()
To define values for our comparisons, we need to wrap the Javascript values in the value() method.
value(val, type = getDefaultType(value))Code language:JavaScript(javascript)
Most of the time you can get away with simply wrapping your JavaScript values with value(). The scalar type is then chosen based on the JavaScript type. If you use special number types or if you need a “Date” you must define that type as second argument.
While the FilterDSL uses prototype chaining to provide the API, it returns plain Javascript objects and arrays.
We use a special Condition scalar to transport these condition graphs through GraphQL without having to select the fields, which we couldn’t anyway except by setting arbitrary complexity limits on our filters.
Sorting
Just like our Condition scalar is a object equivalent to the WHERE clause, we can define the ORDER BY clause by using the sortFields field / the FieldExpression scalar.
The FieldExpression scalar is a close cousin of the condition API with a special shortcut version.
Sort descending by the sum of the fields numA and numB.
Generalized Filtering
The FilterDSL started out and was designed to be transformed into a JOOQ condition and finally an SQL WHERE clause, but in the end it is a pretty abstract filter definition language.
Just like we did in OpenSAGA, we’re using this to filter different things in different contexts.
Automaton contains a transformer that transforms FilterDSL condition graphs into a hierarchy of Java filter classes that can be used to filter Java object instances.
So when we implemented Websocket pubsub for Automaton we could use that to implement a nice topic based pubsub that can execute user-defined filters to decide who wants to see what message on each topic.
So when we have an application of that like the useDomainMonitor hook, we can very succinctly tell the hook what we’re interested in.
Above code will create a monitor instance that receives all domain monitoring meta data for our Foo type. But we could just as well limit that to a single entity or a mix of entity types.
Working with results
In our example above, we receive the iQueryFoo object which is an InteractiveQueryFoo instance.
Now automaton not only does auto-conversion of GraphQL queries to bring the scalar values into the right formats, it also can instantiate MobX classes for GraphQL types and even based on the original type of the degenerified type, which means for InteractiveQueryFoo that we will automatically receive a JavaScript implementation for InteractiveQuery which contains the data fields as observable fields but also offers two methods to continue to work with the query.
will update iQueryFoo with the rows for the second page. Internally the InteractiveQuery.js implementation will just update the document observable with new values.
This allows for cooperative control of data-sources from e.g. a datagrid widget and an external complex filter form.
Injection Pitfalls
It is possible to provide a default filter for both directly and indirectly injected InteractiveQuery documents with one caveat.
babel-plugin-track-usage is not clever enough to understand our FilterDSL so we can’t use it 🙁
If you find yourself in that situation it’s mostly the easiest to write a little script where you can test a FilterDSL expression and then grab the JSON output of that.
InteractiveQuery Configuration
The columnStates field is used for an advanced feature where a user can customize the GraphQL query. The query definition defines the maximum set of fields and the user can disable fields.
This requires the server-side to store e.g. user-specifc configuration and merge that in query method to use instead of the default full selection.
Complex Query Performance
In spite of being very very complex and versatile, we actually managed to get the standard querying mechanism to perform about as well as possible.
We don’t do cascading fetcher fetching, but we also don’t batch like it is commonly done.
Instead, the InteractiveQueryService looks at the domain / GraphQL types and creates an optimized execution plan that fetches the results in very small number of queries.
For one, if we, like in the example about have our Foo with owner field, we don’t fetch the foo object and then the owner, we actually do a
SELECT "foo"."id", "foo"."name", "foo"."description", "foo"."flag", "foo"."type", "owner"."id", "owner"."login", "foo"."owner_id"
FROM "public"."foo"as"foo"
LEFT OUTER JOIN "public"."app_user"as"owner"
ON "owner"."id" = "foo"."owner_id"
ORDER BY "foo"."name"Code language:JavaScript(javascript)
that is we join-in foreign key relations right within the initial query.
Only when we follow *-to-many we split off into different queries which are then batched in themselves. If I fetch ten Foo and go into e.g. a bars field containing a List of associated Bar entities, the InteractiveQuery service will fetch all bar instances with one query.
There’s one limitation in that such a *-to-many “join” will always be a left outer join and not a right join or so.
Outlook
The InteractiveQuery mechanism is something that is really lacking in DomainQL if you should happen to need it. It is however also very opinionated and kind of the one query to rule them all.
It would be one of the top candidates to be moved into its own sub-library.
