Category: Blog

Watercolor Projection process

Painting with light (AI generated)

I thought I document the process I figured out for myself of how to best approach doing watercolor painting based on digital sketches with the help of a DLP projector.

Background

I have been working on my painting skills for some years now and ended up in a on/off relationship with watercolor. Did not like acrylpainting, did a limited amount of gouache painting and kept coming back to watercolor even though it made me quite mad at times.

One of the things I was very dissatisfied with was using pencil to sketch out my paintings. It was always a struggle to draw faint enough so that the markings wouldn’t bother me in the final result and also a lot of losing shapes and lines in the heat of things.

While I made my peace with watercolor not always coming out like you want to, it kept bothering me how I would also lose my sketch and had to start over from zero to redo a painting.

DLP-Projector

I discovered DLP (Digital Light Processing) projectors by accident in a youtube video. I wasn’t even aware that the technology existed and that it allows for devices that are very different from your clunky old projectors. The “digital micro mirrors” basically allow for fine-controlled projection that isn’t straight forward.

I ended up buying a “EKASN Mini Smart DLP Projector” which no longer seems to be avaible, which sadly often seems to happen, but I imagine that other devices based on the same DLP chips will work very similar.

My projector runs “Android TV” as operating system which offers a lot of options of how to get images on there, but in the end, the installed File explorer thing comes with a Browser based upload thing (with Chinese UI 🤣) that seems to work best.

Starting a Painting

When I want to paint a painting, it starts with the paper I want to paint on. I usually work with watercolor paper blocks, so I chose the texture and size I want.

So let’s say I use Arches cotton paper, 31cm x 23cm. That means I create a new image that has a nice big resolution that works well on my Cintiq. Usually I chose the larger dimension to be 1920 which is the FullHD width and also a good fit for the height on my screen with enough room on the left and right for palettes, toolbars and references and whatnot.

So for the Arches 31cm x 23cm in landscape format and a target width of 1920 pixel, we need a height of 1424px (1920 x 23 / 31). I usually tape my paintings to have a nice white border in the end, so I include that in my template and use the area for some simple projection markers.

One of my painting sketches with references

Here I created a border for the painter’s tape I have which is 18mm wide. You can download the empty template in OpenRaster-format below. (I use OpenRaster so I can have a multi-layer image I can open in Krita as well as Gimp).

The example above has multiple sketch layers. For one the handdrawn sketch and then a layer with black at a low opacity where I planned the shadows for the scene.

When I do the shadows, I duplicate the shadows and use an edge filter to get the outlines for it, too. Then I save two different images. One just with the lines and one with the shadow layer.

Preparation

Once I’m ready to start projecting, I save the projection images as PNG and run them through my preparation script.

#!/bin/bash if [ -z $1 ] || [ -z $2 ]; then echo "Usage: c900prepare <image> <output>" exit 1 fi # scale to c900 native resolution, rotated if needed convert $1 -rotate "90<" -scale 1280x720\! $2
Code language: PHP (php)

This is a linux shell script using ImageMagick and the last line is the most important $1 is the input file here and $2 the output file.

If this all sounds to complicated for you, the basic idea of the command is that I scale down my projection images to the exact native resolution of the projector (1280×720). This scaling disregards the correct aspect ratio of the image and stretches it to fit. What the script also does is take care of the orientation. 1280×720 is obviously landscape format, but sometimes I want to project in portrait format that is higher than its wide. So this command rotates the image by 90 degrees if it is portrait format so that the former top of the image is to right which seems to be the easiest way to setup my projection equipment.

Setting up the projection

My watercolor working space

So I have my table top easel where I can adjust the angle, sometimes I work at an angle like this for controlled washes, sometimes I work flat. I bought these photography arms that I assembled to this McGyver-ed configuration. This way I can adjust my arms in all kinds of directions. I usually try have the projector a good length in front of me so that it doesn’t project downwards and I am always shadowing what I paint. By projecting from the front the shadows fall towards me and the area under my brush is as illuminated as possible.

As you can see the projection comes from quite an angle at the paper, but it still perfectly hits the square of it. The manual projection adjustment allows each corner of the projection to be moved pixel by pixel until it matches the canvas.

How to adjust the projector

The projector position should be made to fit the paper as good as possible. Reset the projection if necessary and put the projector at a distance where all four corners of the paper are in the light. Make sure the image is centered on the paper.

Now use the projection adjustment to move the corners inwards until they match the paper. I usually adjust it so that the light creates a bit of shine outside of the paper and then adjust that so that the shine has the same brightness along the whole edge of the paper. This seems to be more precise than trying to have the slightly fuzzy projection edge match the paper edge (Arches paper also has a black binding which makes that even more difficult).

This process fixes the disregard of the aspect ratio in my preparation step as it stretches the projectors native resolution back to the aspect ratio of the paper I started out with. If I did my math right, the lines I draw around the image in my template match exactly with the inner border of my tape and the little projection marker boxes fill out the corners.

Conclusion

I really like to work this way, it allows me to work digitally on my sketches and move things around, scale them, copy and paste parts, whatever, the full digital convenience. If I fail to paint how I want, I can always restart with the same projection sketch. For some of my paintings sometimes I do three or four attempts.

It is no silver bullet. The projected lines are pretty strong if maybe a bit low resolution sometimes. It is still possible to get lost. Especially when I’ve already laid down the darker paint it can be hard to see the lines. I often flip back and forth between the projection image and an empty white image which can help to find lost lines. At some point it is just best to just let go of the projection and just work with the painting. I also have another android tablet for reference images. Mostly for color reference, but I also copy the projection image over to keep working with it while not actually projecting.

References

Inktober 2022

This year I felt like I could do an Inktober again. Life got in the way pretty badly and for a while I felt like doing Inktober was too much, but in the end it felt like it rather helped. In the end I ended up doing 21 of the 31 prompts.

“Day 15: Armadillo” and “Day 27: Snack” totally did not come out as intended. But by my extended Inktober rules, I just left them as-is. Redoing stuff when you’re on a daily schedule is just too much.

The others I think are all pretty good and also show a progression in drawing and composition skills compared to earlier years.

Buffy/Angel Episode Guide

spooky tree

I decided it was time for me to rewatch Buffy and Angel and I wanted to do so in the right order. I found this blog post that gives a recommended order and what episodes are especially important to sync up because they reference the other series.

But the episodes are just listed with consecutive number. That in combination with Disney+ insisting to show me the German titles for the episodes made it necessary to keep counting to know when to jump over etc.

So I made myself another version of the list that shows the common season/episode codes for the episodes.

Links

D&D characters

I often don’t actually post my paintings here, which I totally should. Here are some character paintings I did people in /r/characterdrawing

Form Design in domainql-form

In this post I will try to go through high-level design aspects of domainql-form. How I initially envisioned them to work and how they evolved over time driven by our experiences with it.

First concept

The first concept was really simple. We have a MobX domain object that represents the root object for the part of the domain we manage with the form, and then we have the <Form/> component that receives the object as value prop.

The <Field/> components reference paths within the object graph by lodash like paths. Here for our example “owner.name”, which first references the owner object that is embedded as property within the root object and then the name property within that owner object.

I mostly imagined one big <Form/> component per view.

Diagram showing the initially envisioned connection between the Form component and MobX domain object.

This works and for large parts still works this way, but over time we slowly evolved into a more complex model.

Problem: HTML

The first thing that became on issue was the of course well-known fact that you can’t nest forms within HTML. And at first it seems like, duh, who would do something like that? But then you get a use-case where you have some form fields and then data-table connected to that object and then some more form fields. And of course, the <Datagrid/> needs to do all kinds of form control related things like choosing pagination sizes, letting the user enter filter values etc pp.

Problem: Awkard forms

While in some cases it is just natural to have sub-ordinate objects connected in the form and to edit fields within those, but as soon as you get to lists or deeply nested property paths it gets awkward very quickly, especially if you have to construct the paths dynamically.

Solution: Many Forms Paradigm

So we obviously need to be able to have many forms, often referencing the same object. But we also have cases where we want to edit the nth element out of a list of associated entities.

The form components now point anywhere they like. One root object, many root objects, objects within root objects, doesn’t matter.

Diagram showing the new "Many Forms" approach

We just have many forms that write into the same (non-isolated) objects. These <Form/> components all have their own <form/> elements. But what we want most of the time is that the forms behave as if they were one form.

If the user has entered erroneous data and there is an error displayed, all non-discarding buttons must be disabled. Only things like “Cancel” can remain enabled.

FormContext

This orchestration of <Form/> component functionality is handled by the new FormContext class. There is a default context that is always used unless the application author created and referenced another FormContext. e.g. For processes that have to independent form-flows side-by-side or a main process and a sidebar-process.

The FormContext also registers all available memoized field-contexts which can be used to implement high-level form behavior on top of domainql-form.

Wacom Cintiq Pro on Ubuntu 20.04 focal fossa

Pretty CC-picture of a hummingbird

This post is meant to collect my experiences with my new Wacom Cintiq graphics tablet in Ubuntu and document the current state of my configuration for posterity so that it may help others. It contains my Express Keys setup for Krita.

Setup

The documentation leaflets provided by Wacom seemed suboptimal to me. It was some IKEA style no-words-only-pictures thing that did not make quite clear to me which part was which.

In the end, I just decided to plug it in like it seemed right. And after connection power, HDMI plug and the USB-C cable for the input devices the whole thing worked perfectly. Graphics tablet was recognized with the right resolution, its own color profile and everything. I just needed to tell Ubuntu that his monitor is supposed to be below my normal one. I also had deactivated the key combinations to move windows between monitors in my Ubuntu configuration, because I never needed it and thus had to restore that.

Express-Keys Configuration

At first I was disappointed that the pinch zoom gesture worked out-of-the-box in Krita, but I couldn’t get a multi-touch rotation bound to Krita’s canvas rotation.

Krita already very wisely prevents you from accidentally touch-painting on your graphics tablet. But in the end, the few times where it was nice to select tools with the non-dominant hand seemed not to worth it compared to the many times I inadvertently activated other parts of the UI like tool selections or the scroll bars. So I disabled touch for the graphics tablet.

Diagram showing my current express key setup (source code below)

My goal was to have the most common functionality I need for my paint workflow on the Express keys so that I can work without keyboard for larger stretches without moving between pen/express keys and keyboard/mouse operation too much.

The touch wheel does the canvas rotation and the mode-switch in the center just does a reset, which is a combination of resetting canvas rotation, resetting zoom and setting the tool to “Brush”.

Ring left and right are undo and redo, Ring down resets zoom only.

The left/right button pairs below the ring control brush size, opacity and brightness. The last two are less important to my paint workflow, but they are still useful sometimes and I have enough buttons.

The left/right pair on the bottom are pure modifier keys. Pressing control while in Brush mode activates the (mixing) color picker, but there’s a lot of functionality where ctrl and shift are modifiers in Krita.

The commands to set up this configuration are as follows:

#!/usr/bin/bash xsetwacom --set "Wacom Express Key Remote Pad pad" Touch on # experimental total touch prevention xsetwacom --set "Wacom Cintiq Pro 32 Touch Finger touch" Touch off # settings for krita (with some extra key configuration there) # zoom (ctrl +/-) (not in diagram) xsetwacom --set "Wacom Express Key Remote Pad pad" Button 12 "key ctrl plus" xsetwacom --set "Wacom Express Key Remote Pad pad" Button 15 "key ctrl minus" # rotation over touch wheel (ctrl ö/ä) xsetwacom --set "Wacom Express Key Remote Pad pad" AbsWheelUp "key ctrl 0xd6" xsetwacom --set "Wacom Express Key Remote Pad pad" AbsWheelDown "key ctrl 0xe4" # wheel button left/right for undo/redo xsetwacom --set "Wacom Express Key Remote Pad pad" Button 2 "key ctrl z" xsetwacom --set "Wacom Express Key Remote Pad pad" Button 9 "key ctrl shift z" # Mode resets rotation and tool to brush and zoom (no modes yet) xsetwacom --set "Wacom Express Key Remote Pad pad" Button 1 "key 5 b 1" # wheel down: reset zoom only xsetwacom --set "Wacom Express Key Remote Pad pad" Button 10 "key 1" # 1st row: Brush size ö/ä xsetwacom --set "Wacom Express Key Remote Pad pad" Button 11 "key +0xd6" xsetwacom --set "Wacom Express Key Remote Pad pad" Button 13 "key +0xe4" # 2nd row: opacity xsetwacom --set "Wacom Express Key Remote Pad pad" Button 14 "key +i" xsetwacom --set "Wacom Express Key Remote Pad pad" Button 16 "key +o" # 3rd row: color brightness xsetwacom --set "Wacom Express Key Remote Pad pad" Button 17 "key +k" xsetwacom --set "Wacom Express Key Remote Pad pad" Button 19 "key +l" # bottom buttons: ctrl + shift xsetwacom --set "Wacom Express Key Remote Pad pad" Button 21 "key +ctrl" xsetwacom --set "Wacom Express Key Remote Pad pad" Button 22 "key +shift"
Code language: PHP (php)

As you can see, I put some functionality on the “ö” and “ä” keys that come with my German keyboard and which are always useful keys because they’re almost always free in the default key bindings.

Without a German keyboard, you obviously want other combos there. I think the ö/ä combos here are also the ones where I configured the extra key configuration in Krita to have them. All others are default keys I think.

The source code contains two zoom combos that are not in the diagram and which I might remove completely. I mostly try to work at 1:1 resolution on the graphics tablet anyway and rarely zoom in for details — and there’s always mouse wheel zoom.

Update: In the end I decided to buy the Wacom Flex arm which kind of made all this superfluous. Now I tend to just move the pad into a vertical-ish position slightly to the right so I can rotate my chair and work and have my full keyboard accessible on the left.

Darkly, free

I was looking for a good looking dark theme for one of my toy projects and came across a bootstrap theme called “darkly” which looks nice but as so often, comes with a one of those snoopy Google fonts that can/do track you etc pp.

In this case it is even more annoying, because the actual font “Lato” is an open font.

So here it is: a version of the darkly bootstrap 4 theme with the font files to serve from the same server. The CSS uses relative addressing, i.e. the “css” and “webfonts” folder must be siblings on your server

Toggle Show WireFrame Addon for Blender

Recently watched a youtube video about the Blender boolean workflow.

Even though I got the Boxcutter and HardOps addons, they’re basically just a fancier and more user-friendly repackaging of the boolean operation workflow in Blender without those addons. So I thought there might be value in understand the normal one better.

But that’s not the topic here. While he was explaining things, he mentioned that he made a keyboard shortcut for an option that I always found more attractive than the comparable alternatives.

If you click “Wireframe” under “Viewport Display” in the Object properties tab, you get the same render mode as before, but with a wireframe of the object on top.

In find this option much more attractive than just enabling Wireframe for all objects. You can fine control which objects you want to see and don’t suddenly have a huge mess of lines from all the objects.

Most often, I just want to check two meshes against each other.

Image showing the location of the wireframe option within the Blender UI

However, that option there is no Blender operation, so you can’t define a keyboard shortcut for it :\

However, after some googling I found that it is easy to write your own addon to add such an operation with a default key binding and all.

Install Addon

To install the addon, download and save the python file below and then click “Install…” in the Blender preferences under “Add-ons” and select the file to install it.

Image showing the location of the "Install..." button within the Blender Preferences

Usage

With the addon installed, you can select one or multiple objects in the object mode and press Alt + Shift + J to toggle the wireframe display for all selected objects individually.

If you don’t like the default shortcut, you can change it in the Blender preferences under “Keymap”.

Links

InteractiveQuery in Automaton

For the second post in our “Cool shit in Automaton” series, I’d like to write about the InteractiveQuery mechanism. From the general conception to the final implementation with some nice Automaton features.

This post might be conceptually interesting for everyone involved with GraphQL, concrete it is most useful if you’re using Automaton.

Design

The InteractiveQuery mechanism was designed to drive the data needs of our widgets. It is a set of high-level abstract query definition types that offer interactive pagination, filtering, sorting and configuring of GraphQL queries.

It touches many Automaton subsystems which I will discuss in some detail here although it is not strictly necessary to understand all of it to use it. In many ways it’s pretty magical.

Defining an InteractiveQuery based GraphQL Query

One of design goals was of course user-friendliness but not at the expense of power and general applicability. A part of that being that we want to write the least amount of code possible, but still retain all the flexibility we need in our client’s domains.

InteractiveQuery turned out to be the driving force behind the Degenerification feature in DomainQL. The basic idea is that we want to be able to use Generics in Java and translate that into the GraphQL type world.

Let’s look at the default GraphQL query implementation for an example entity.

/** * Default implementation of an InteractiveQuery based query for type [T]. * * @param type * @param env * @param config configuration for the Interactive query. * @param <T> * @return */ @GraphQLQuery public <T> InteractiveQuery<T> iQuery( @GraphQLTypeParam( types = { Foo.class } ) Class<T> type, DataFetchingEnvironment env, @NotNull QueryConfig config ) { log.info("iQuery<{}>, config = {}", type, config); return interactiveQueryService.buildInteractiveQuery( type, env, config) .execute(); }
Code language: Java (java)

This is the standard implementation for InteractiveQuery based GraphQL queries. The first method parameter “type” controls the generation of types for this generic method. For each concrete type listed, a degenerified variant of the iQuery method will be defined, by default appending the name of the concrete type to the method method name (here “iQueryFoo”). This is not the most pretty solution but it does keep things in order especially with huge domains. Our app schema already contains 42 variants of this and we’ve just started.

InteractiveQuery in GraphQL

Let’s take a look at the GraphQL types generated from that Java code snippet.

type QueryType { "Default implementation of an InteractiveQuery based query for type Foo." iQueryFoo(config: QueryConfigInput!): InteractiveQueryFoo } "Interactive Query with Foo payload." type InteractiveQueryFoo { "Column states for the current result." columnStates: [ColumnState] "Query configuration the current result was produced with." queryConfig: QueryConfig "Total row count available." rowCount: Int "List with current rows of Foo." rows: [Foo] "Name of payload type (always 'Foo')" type: String } "The state of a column within an interactive query." type ColumnState { "True if column is enabled. Server might disabled columns." enabled: Boolean "Column name" name: String "True if the column is sortable." sortable: Boolean } "Encapsulates all parameters of an interactive query." type QueryConfig { "FilterDSL condition graph or null" condition: Condition "Current page within the paginated results" offset: Int "Optional unique query identifier. Useful for server-side query implementations." id: String "Maximum number of paginated results.," pageSize: Int "Current sort order for the query." sortFields: [FieldExpression] } "Map graph representing JOOQ conditions" scalar Condition "Map graph representing a JOOQ field expression" scalar FieldExpression
Code language: JavaScript (javascript)

The iQueryFoo query returns the InteractiveQueryFoo type which is a degenerification of InteractiveQuery<T> for the type Foo.

The InteractiveQuery types are all structurally the same and only differ in the payload type. The “rows” field contains a List of the payload type.

Filtering and Sorting

Here is where the real magic starts. So we want to filter a query, but we don’t want to write code for it. The widget provides a JSON description of a filter which we then apply in the context of the current SQL query.

FilterDSL

There is another complication in that while we / Automaton clearly prefers PostgreSQL as database, our clients may or may not, so we will support all databases that JOOQ supports.

Which also means that we wanted to avoid mucking around with SQL in that degree in any case, i.e. we needed an actual abstraction.

Luckily, JOOQ comes with it’s own Condition API which we can just adapt.

So we created the FilterDSL which is a pretty close copy of the JOOQ condition DSL with some necessary additions.

./src/main/js/apps/myapp/queries/Q_Foo.js
import { query } from "@quinscape/automaton-js" export default query( // language=GraphQL `query iQueryFoo($config: QueryConfigInput!) { iQueryFoo(config: $config) { type columnStates{ name enabled sortable } queryConfig{ id condition currentPage pageSize sortFields } rows{ id name description flag type owner{ id login } } rowCount } }`, { "config": { "pageSize": 20 } } )
Code language: JavaScript (javascript)

Here we see the query definition to query our Foo type. It has fields of different scalar types and an embedded owner object with an id and a login name.

Now let’s define a filter for that query

import { FilterDSL } from "@quinscape/automaton-js"; import Q_Foo from "../../queries/Q_Foo"; // deconstruct FilterDSL methods const { field, value, and, or, not } = FilterDSL; Q_Foo.execute({ config: { condition : and( field("name").eq(value("AAA")), not( field(value("owner.login")).eq("admin") ) ) } }) .then( ({iQueryFoo}) => { // ... } )
Code language: JavaScript (javascript)

We query all Foo objects whose name is “AAA” and which are not owned by “admin”.

You can choose how you want to write your boolean conditions. Either like above or you could write the same condition as

field("name").eq( value("AAA") ).and( field( value("owner.login") ).eq("admin") .not() )
Code language: CSS (css)

or a mix in between. Personally I find the second style not so good and the dangling .not() is outright horrible.

Using value()

To define values for our comparisons, we need to wrap the Javascript values in the value() method.

value(val, type = getDefaultType(value))
Code language: JavaScript (javascript)

Most of the time you can get away with simply wrapping your JavaScript values with value(). The scalar type is then chosen based on the JavaScript type. If you use special number types or if you need a “Date” you must define that type as second argument.

While the FilterDSL uses prototype chaining to provide the API, it returns plain Javascript objects and arrays.

For our example condition the JSON would be

{ "type": "Condition", "name": "and", "operands": [ { "type": "Condition", "name": "eq", "operands": [ { "type": "Field", "name": "name" }, { "type": "Value", "value": "AAA", "scalarType" : "String", "name": null } ] }, { "type": "Condition", "name": "not", "operands": [ { "type": "Condition", "name": "eq", "operands": [ { "type": "Field", "name": "owner.login" }, { "type": "Value", "value": "admin", "scalarType" : "String", "name": null } ] } ] } ] }
Code language: JSON / JSON with Comments (json)

We use a special Condition scalar to transport these condition graphs through GraphQL without having to select the fields, which we couldn’t anyway except by setting arbitrary complexity limits on our filters.

Sorting

Just like our Condition scalar is a object equivalent to the WHERE clause, we can define the ORDER BY clause by using the sortFields field / the FieldExpression scalar.

The FieldExpression scalar is a close cousin of the condition API with a special shortcut version.

Most commonly we want very simple sorting

sortFields: ["name"]
Code language: CSS (css)

Sort by name

sortFields: ["owner.login", "!name"]
Code language: CSS (css)

Sort first by owner name and then descending by name.

And because the implementation was nearly trivial, we can also do complex sorting.

sortFields: [ field("numA").add(field("numB")).desc() ]
Code language: CSS (css)

Sort descending by the sum of the fields numA and numB.

Generalized Filtering

The FilterDSL started out and was designed to be transformed into a JOOQ condition and finally an SQL WHERE clause, but in the end it is a pretty abstract filter definition language.

Just like we did in OpenSAGA, we’re using this to filter different things in different contexts.

Automaton contains a transformer that transforms FilterDSL condition graphs into a hierarchy of Java filter classes that can be used to filter Java object instances.

So when we implemented Websocket pubsub for Automaton we could use that to implement a nice topic based pubsub that can execute user-defined filters to decide who wants to see what message on each topic.

So when we have an application of that like the useDomainMonitor hook, we can very succinctly tell the hook what we’re interested in.

const monitor = useDomainMonitor( field("domainType") .eq( value("Foo") ) )
Code language: JavaScript (javascript)

Above code will create a monitor instance that receives all domain monitoring meta data for our Foo type. But we could just as well limit that to a single entity or a mix of entity types.

Working with results

In our example above, we receive the iQueryFoo object which is an InteractiveQueryFoo instance.

Now automaton not only does auto-conversion of GraphQL queries to bring the scalar values into the right formats, it also can instantiate MobX classes for GraphQL types and even based on the original type of the degenerified type, which means for InteractiveQueryFoo that we will automatically receive a JavaScript implementation for InteractiveQuery which contains the data fields as observable fields but also offers two methods to continue to work with the query.

// update the iQuery document iQueryFoo.update(queryConfig) // Update / merge query conditions (advanced usage) iQueryFoo.updateCondition( componentCondition, componentId = NO_COMPONENT, checkConditions = true )
Code language: JavaScript (javascript)

iq.update(config) let’s you update the iQuery document with a new partial query config. For example

iQueryFoo.update({ offset: 20})
Code language: CSS (css)

will update iQueryFoo with the rows for the second page. Internally the InteractiveQuery.js implementation will just update the document observable with new values.

This allows for cooperative control of data-sources from e.g. a datagrid widget and an external complex filter form.

Injection Pitfalls

It is possible to provide a default filter for both directly and indirectly injected InteractiveQuery documents with one caveat.

babel-plugin-track-usage is not clever enough to understand our FilterDSL so we can’t use it 🙁

If you find yourself in that situation it’s mostly the easiest to write a little script where you can test a FilterDSL expression and then grab the JSON output of that.

InteractiveQuery Configuration

The columnStates field is used for an advanced feature where a user can customize the GraphQL query. The query definition defines the maximum set of fields and the user can disable fields.

This requires the server-side to store e.g. user-specifc configuration and merge that in query method to use instead of the default full selection.

Complex Query Performance

In spite of being very very complex and versatile, we actually managed to get the standard querying mechanism to perform about as well as possible.

We don’t do cascading fetcher fetching, but we also don’t batch like it is commonly done.

Instead, the InteractiveQueryService looks at the domain / GraphQL types and creates an optimized execution plan that fetches the results in very small number of queries.

For one, if we, like in the example about have our Foo with owner field, we don’t fetch the foo object and then the owner, we actually do a

SELECT "foo"."id", "foo"."name", "foo"."description", "foo"."flag", "foo"."type", "owner"."id", "owner"."login", "foo"."owner_id" FROM "public"."foo" as "foo" LEFT OUTER JOIN "public"."app_user" as "owner" ON "owner"."id" = "foo"."owner_id" ORDER BY "foo"."name"
Code language: JavaScript (javascript)

that is we join-in foreign key relations right within the initial query.

Only when we follow *-to-many we split off into different queries which are then batched in themselves. If I fetch ten Foo and go into e.g. a bars field containing a List of associated Bar entities, the InteractiveQuery service will fetch all bar instances with one query.

There’s one limitation in that such a *-to-many “join” will always be a left outer join and not a right join or so.

Outlook

The InteractiveQuery mechanism is something that is really lacking in DomainQL if you should happen to need it. It is however also very opinionated and kind of the one query to rule them all.

It would be one of the top candidates to be moved into its own sub-library.

Links

GraphQL-injection with Automaton/DomainQL

Symbol picture: Injection needle with GraphQL-Logo

In this blog post I will try to explain how we organize data loading in Automaton/DomainQL and walk you through the general idea of it and how to adapt it for your own project.

Conceptually it provides an alternative to asynchronous loading / suspense on application startup.

Background

We are working on the relaunch of a German state-level government application. We have been doing paid Open-Source development for clients for quite some time now and are now doing it again for this project as requested by our client.

On the most general level, we use DomainQL to drive our GraphQL.

Symbolic DomainQL Explanation: Database + JOOQ + DomainQL is combined to GraphQL

The current workflow is that we start with a PostgreSQL database from which JOOQ generates Java classes, so called POJOs (Plain Old Java Objects, basically normal/anemic Java classes with getters and setters according to the old Java Bean standard).

Basic injection in DomainQL

DomainQL already contains a comparatively simple GraphQL injection method based on exports with magic names. At point I’m close to considering it deprecated since the new system works so much better now.

At first I expected DomainQL to be our main thing, but along the way it became clear to me that we needed more and we added Automaton as an additional layer. Where DomainQL is still very general, Automaton has a clear and distinct vision and a very opinionated way of getting there.

GraphQL injection in Automaton

Let’s start with the basic idea of our kind of data injection. In a traditional Javascript application, the user loads the initial HTML document which contains very little, it downloads all the Javascript bundles and CSS bundles and other resources and then the JavaScript is executed and let’s say React starts rendering an application.

For which it might need a lot of data which it loads asynchronously which necessitates loading indicators, which prompted solutions like Suspense etc.

Often, this is preferable to having the user wait longer but it introduces a lot of additional latency, even if we can manage to tame the visual artifacts with Suspense.

But wouldn’t it be awesome if the components didn’t have to request the data they need because it is already there? What if the server could provide all the data the components need by knowing what they will request before they do?

Static Code Analysis

When I started wondering about how to do this, I was already using the solution, a small, maybe odd library I had written earlier called babel-plugin-track-usage.

Code is just data and as we do it, we never actually use JavaScript as-is, but always transpile it, compress it etc.

babel-plugin-track-usage uses the Babel AST to detect and track statically analyzable calls within your code base.

At first we used it for i18n.

[ "track-usage", { "sourceRoot" : "src/main/js/", "trackedFunctions": { "i18n": { "module": "./service/i18n", "fn": "", "varArgs": true } }, "debug": false } ]
Code language: JSON / JSON with Comments (json)

Here we see the plugin configuration for an i18n service. We define that all our sources are under “src/main/js” (we’re using a Maven project layout) and then we define that we want to track the default export of a module “./service/i18n” which accepts variable arguments.

This lets us write expressions like “i18n("Hello {0}", user)” in our code. The function just looks up the first argument in a translation map that contains all the necessary translations because the babel-plugin-track-usage provided the server with all the invocations of our i18n function as long as all arguments are statically analyzable. With the varArgs setting it only considers the first argument and allows non-static analyzable varargs to follow.

In combination with a mini-Webpack-Plugin, babel-plugin-track-usage then exports JSON data for the server to process.

{ "usages": { "./apps/shipping/processes/crud-test/composites/CRUDList": { "requires": [ "@quinscape/automaton-js", "mobx-react-lite" ], "calls": { "i18n": [ ["Foo List"], ["Action"], ["owner"] ] } }, ... } }
Code language: JavaScript (javascript)

“usages” contains a map of relative module names as keys. The “calls” array contains a map of symbolic call names (Here our “i18n”) with a nested array containing an array per function invocation found.

“requires” contains an array with all modules required by that module. This can be used to follow all import dependencies and find all calls starting with a module as starting point.

After I extended the library a few times to support template literals, ES6 imports and also identifiers, it was ready to drive our data injection too.

Direct GraphQL Query injection

Schematic diagram illustrating the injection process explained below

Our apps in automaton correspond to Webpack entrypoints. Each of them contains a number of processes that make up the application functionality.

For each process there is a MobX class that defines the process scope containing all the data for the process.

Here is how our first version of GraphQL injection looked and still does, although we rarely use it like this

import { observable } from "mobx"; import { query } from "@quinscape/automaton-js" export default class ExampleScope { @observable injected = query( // language=GraphQL `query iQueryFoo($config: QueryConfigInput!) { iQueryFoo(config: $config) { # ... } }`, { "config": { "pageSize": 5, "sortFields": ["name"] } } ) }
Code language: JavaScript (javascript)

We important the “query” function from our library which is registered tracked. The function takes a simple template literal string with the query and a static JSON block with the default config as second argument.

The naming conventions for our file tell the server which process the statically analyzable query belongs to. We follow all imports etc.

When the user starts the application with one of its processes, the server can look up the query() calls it found and use that to fetch the data from GraphQL service.

This method works very well, even when we inject many queries. We don’t even need to bother to craft GraphQL documents with multiple queries. All injected queries will be executed together using an internal shortcut without using HTTP. (Hurray for GraphQL’s transport neutrality).

At the end we can then embed all the queried data in our initial HTML object as script tag with fantasy type.

Our system then can provide the process with a new instance of the process scope that magically contains all the injected data blocks and the React components involved can start rendering right away.

The main reason we switched to a different method is that we in fact have a lot of queries to execute in some processes which makes the process scope very lengthy

Indirect GraphQL injection

At first I couldn’t quite figure out an elegant solution until I realized that like so often, the solution to the problem is another layer of indirection.

What we did was to reify the query arguments into its own named thing and add a second tracked function.

We split the injection into multiple files.

First we have the query definition which we put into its own file.

./src/queries/Q_Foo.js
import { query } from "@quinscape/automaton-js" export default query( // language=GraphQL `query iQueryFoo($config: QueryConfigInput!) { iQueryFoo(config: $config) { # ... } }`, { "config": { "pageSize": 5, "sortFields": ["name"] } } )
Code language: JavaScript (javascript)

We have a standard folder where all our named query definitions are located (here shortened to ./src/queries/). Each definition is named after the file it is in, here Q_Foo.

Now to inject the current value of Q_Foo we just need to write

./src/processes/example/example.js
import { observable } from "mobx"; import { query, injection } from "@quinscape/automaton-js" import Q_Foo from "../../queries/Q_Foo" export default class ExampleScope { @observable injected = injection(Q_Foo) }
Code language: JavaScript (javascript)

I just love the way it works out. The Javascript syntax is just like one would normally write it, I can use my IDE to ctrl+click on the the name and lookup the definitions (or do quick lookups etc). Everything is neatly organized and the scope definitions are much clearer.

The plugin just generates a special JSON block for our identifier

{ "usages": { "./src/processes/example/example": { "requires": [ "mobx", "@quinscape/automaton-js", "./src/queries/Q_Foo" ], "calls": { "injection": [ [ { "__identifier": "Q_Foo" } ] ] } } } }
Code language: JSON / JSON with Comments (json)

The JSON data just provides the name and the server has to know / define what that means, in our case “Hey there should be a ‘./src/queries/Q_Foo’ which in turn contains a query() invocation that defines what data we want here.”

Links