If you think GatsbyJS is only a static site generator, you are missing the point

By Alex Lobera17 Dec 2019
Reading time: 8 mins

On this page:

After a year using GatsbyJS to build what I thought was a “simple static site”, I had a kerpow: GatsbyJS is NOT a static site generator, but an asynchronous content mesh system for the modern Web, powered by GraphQL. There were two concepts that together helped me realize that: content & time.

For those who don’t know what having a kerpow means (it’s a concept I’ve only come across recently), it means, in two simple words, “holy shit". Let me show you the path that brought me to the realization that I was not simply building a static site.

We see the world from our perspective at a particular point in time

If I look at how I’ve spent my time while using GatsbyJS during the last year, the most time (approximately 70%) was spent looking at JSX code, followed by looking at JSON (data), followed by looking at GraphQL queries, and finally, the least amount of time was spent looking at Node.js code. Bear in mind that this is not a very rigorous 1-year analysis.

Distribution of code & time I’ve been dealing with while using GatsbyJS in the last year.

If your diagram would look similar to mine, you might also be under the impression that GatsbyJS is a static site generator. I run the following Twitter poll about other devs spend their time using Gatsbyjs:

Whatever the case, feel free to tweet me (@alex_lobera) if you want to share your experience :)

What was my main focus this year? We wanted our users to have the best user experience and we iterated this process for one year.

Interestingly, the more I reduce the span of time I analyse, and get closer to the present, the more the pyramid shifts. If I look at the past few weeks, the pyramid looks like this:

What has changed? I’ve focused on content, which gave me a new perspective. I believe our capacity to come to a realisation of something correlates to our exposure to it. Let me share with you what I’ve been exposed to recently, and how it made me change the way I think about GatsbyJS.

Websites as a content mashup

Websites as a content mashup is not a new thing. Ten years ago I was creating “widgets” using JSONP to display content from different sources on a website. The mashup concept has even made it to a prestigious dictionary:

In the past, the mashup was focused on the browser. Why? Because cross-domain requests were complicated, and we developers like complicated things. Just kidding 😜, I don’t know why 😄. What I do know is that the world has changed since then. For instance:

  • There are many more APIs to pull content from.
  • We’ve got better tooling to create that mashup.
  • Ten years ago we had mainly desktops on the client and relatively expensive servers. Now we have a huge variety of devices and different types of networks, as well as cheap cloud computing, compared to the on-premise data centres we used to have to use.

An observation from reading the Cambridge Dictionary. It defines a mashup as using “from different websites”. I wouldn’t say that a headless CMS is a website, and neither is gatsby-image. So these days that definition doesn’t quite fit. The modern term is content mesh.

What I find very interesting is that now the mashup content mesh is created in different places, at different points in time. For instance, this page https://reactgraphql.academy/react/training/bootcamp/london/ uses two stages to pull data: 1) build time, and 2) client time.

Build time

This is what you’ll see if you disable JavaScript on the browser and navigate to this link. What you see is the data that was pulled at build time, which is everything but the price. You see a landing page that explains a training, but you can’t actually buy the training instance yet. If you click on “Buy tickets” with JS disabled you’ll see the following:

No matter how long you wait, no JavaScript, no price.

Client time

This is what you’ll see if you enable JavaScript on the browser, navigate to the same page, and click on the “Buy tickets” button at the top of the page. The browser will scroll down to the price section, and then you’ll be able to see the price at that point in time. The reason fetching the price is delayed until run-time being that the price depends on certain conditions, such as sales promotions, and if the course is sold out, or cancelled, there might be no tickets on sale.

The following pyramid is an approximation of my time spent on building that page

GraphQL as the content mesh composition tool

Things changed when we decided to integrate this blog to a headless CMS. Prior to that, writing an article was a matter of writing some markdown, and merging a PR into master. The CI would then do the rest:

This blog post you are reading right now was created and published using Sanity.io (our headless CMS of choice). This CMS adds the following task to our workflow: posts are published on Sanity.io, and is subsequently available via an API. To be published doesn’t mean it’s displayed on the website. How do we display that post on the website? GatsbyJS plugins.

GatsbyJS plugins, the building blocks we can configure AND extend

GatsbyJS plugins are a very powerful way to source and/ or transform almost any kind of data in a GatsbyJS website. All the data in GatsbyJS become nodes in a graph. By using GraphQL to expose all the data GatsbyJS makes it very easy to query. Being able to easily query data is just the tip of the iceberg. GraphQL is a powerful mechanism to compose and extend data across all the plugins.

First, let me show you what I did to make all my CMS data available on my website. We can connect a GatsbyJS website to Sanity.io using the gatsby-source-sanity plugin. The following 5 lines of code gave me access to all my Sanity content at build time:

But I encountered a problem, that plugin didn’t have a feature I needed. I wanted to download images from Sanity.io at build time to my local so I could distribute them using my CDN of choice. Contentful (another headless CMS) provides a `localFile` field in their schema that implements that.

query MyQuery {
    # Example is for a `ContentType` with a `ContentfulAsset` field
    # You could also query an asset directly via
    # `allContentfulAsset { edges{ node { } } }`
    # or `contentfulAsset(contentful_id: { eq: "contentful_id here" } ) { }`
    contentfulMyContentType {
      myContentfulAssetField {
        # Direct URL to Contentful CDN for this asset
        file { url }

        # Query for a fluid image resource on this `ContentfulAsset` node
        fluid(maxWidth: 500){
          ...GatsbyContentfulFluid_withWebp
        }

        # Query for locally stored file(e.g. An image) - `File` node
        localFile {
          # Where the asset is downloaded into cache, don't use this
          absolutePath
          # Where the asset is copied to for distribution, equivalent to using ContentfulAsset `file {url}`
          publicURL
          # Use `gatsby-image` to create fluid image resource
          childImageSharp {
            fluid(maxWidth: 500) {
              ...GatsbyImageSharpFluid
          }
        }
      }
    }
  }

https://www.gatsbyjs.org/packages/gatsby-source-contentful/#download-assets-for-static-distribution

Unfortunately, Sanity.io didn’t have that field in their GraphQL schema. One solution to this problem is to switch CMS. The problem with switching to another CMS is that it might take a considerable amount of effort and it doesn’t guarantee that the new one will provide all the functionality that I might need in the future. A better solution would be to extend the GraphQL schema and implement the use case I need. These were my requirements:

  • Crop an image using Sanity cloud image service.
  • Download the image from the Sanity CDN to my local machine at build time.
  • Pipe the image file, downloaded at build time, to an ImageSharp type to get an ImageSharpFluid version of it.
  • Pass the ImageSharpFluid data to a gatsby-image React component.
  • Deploy images to my CDN of choice at build time.
  • Download images from my CDN of choice at client time.

You might get lost with the terms and think it’s a lot of work, just bear with me for a second. My distribution of time/type of code to implement that:

We can map that pyramid to the following code:

JSX, 0 props or JSX updated: https://github.com/reactgraphqlacademy/reactgraphqlacademy/blob/master/src/components/blog/PostCard.js#L24

JSON, manipulated data:

 if (fluidImage) {
    imageProps.fluid = fluidImage
  } else {
    imageProps.src = imageUrl
  }

https://github.com/reactgraphqlacademy/reactgraphqlacademy/blob/master/src/components/blog/PostCard.js#L16

GraphQL, updated query:

 fragment SanityPostItemFragment on SanityPost {
    title
    excerpt
    category
    mainImage {
      asset {
        localFile(width: 500, height: 333) {
          publicURL
          childImageSharp {
            fluid {
              ...GatsbyImageSharpFluid
            }
          }
        }
      }
    }
    slug {
      current
    }
  }

https://github.com/reactgraphqlacademy/reactgraphqlacademy/blob/master/src/components/blog/PostCard.js#L48

Node.js code:

To me, this is the most mind-blowing part. Only 61 lines of code (considering that we are using the verbose GraphQL JS reference implementation) to solve the whole problem by creating a plugin that extends another plugin (gatsby-source-sanity) using a third plugin (gatsby-source-filesystem) 🤯.

import { GraphQLInt, GraphQLString } from "gatsby/graphql";
import { createRemoteFileNode } from "gatsby-source-filesystem";

export const setFieldsOnGraphQLNodeType = ({ type }) => {
  if (type.name === `SanityImageAsset`) {
    return {
      localFile: {
        type: `File`,
        args: {
          width: {
            type: GraphQLInt,
            defaultValue: 600
          },
          format: {
            type: GraphQLString,
            defaultValue: "jpg"
          },
          height: {
            type: GraphQLInt
          },
          fit: {
            type: GraphQLString,
            defaultValue: "crop"
          }
        }
      }
    };
  }

  // by default return empty object
  return {};
};

export const createResolvers = ({
  actions: { createNode },
  cache,
  createNodeId,
  createResolvers,
  store,
  reporter
}) => {
  const resolvers = {
    SanityImageAsset: {
      localFile: {
        resolve: (source, { width, height, fit, format }) => {
          return createRemoteFileNode({
            url: `${source.url}?w=${width}&fm=${format}${
              height ? `&h=${height}` : ""
            }&fit=${fit}`,
            store,
            cache,
            createNode,
            createNodeId,
            reporter
          });
        }
      }
    }
  };
  createResolvers(resolvers);
};

https://github.com/reactgraphqlacademy/gatsby-source-sanity-transform-images/blob/master/src/gatsby-node.js.

Asynchronous content mesh system for the Web

I submitted my plugin to the GatsbyJS plugin library following the GatsbyJS docs steps. Soon after I typed “sanity” on their plugin search input and there it was! Mmm, or it wasn’t?

Then I read in their docs: “Algolia will take up to 12 hours to add it to the library search index (the exact time necessary is still unknown), and wait for the daily rebuild of https://gatsbyjs.org to automatically include your plugin page to the website.”

Algolia is a Software as a Service (SaaS) product that specializes in search and discovery. It took about an hour for Algolia to index my plugin. So after an hour, the plugin showed up in the search result. It took over a day for the GatsbyJS build to run again, and finish. That time in between the Algolia index and the GatsbyJS build I was in limbo, the technical name for which is 'eventual consistency’.

That’s not unique to Gatsbyjs.org. We have a similar issue in this blog. When we publish a blog post in Sanity.io, the post is immediately available on the API. Then we need to build the site. The time in between publishing and building, the editor is in limbo.

We can use webhooks to solve the issue. An editor publishes a document and then a webhook triggers a build. Unfortunately, I don’t think that’s currently cost-effective at scale.

With the battle for content very intense right now between so many players, maybe it’s time to start focusing on the next frontier: incremental build on the cloud.

Share this on:


This website is built using Gatsbyjs. Curious about how this blog is implemented? It's open source so you can check the source code


Comments? Shoot me a tweet @alex_lobera !

Upcoming - All Events#

19
Feb3 days

React Fundamentals
London, UK
Prices & more details

19
Feb1 week

React Redux Bootcamp
London, UK
Prices & more details

24
Feb1 day

Advanced React Patterns, FP & Performance
London, UK
Prices & more details

24
Feb3 days

Advanced React Training
London, UK
Prices & more details

25
Feb1 day

Advanced React UIs & Styling With Design Systems
London, UK
Prices & more details

26
Feb1 day

Testing in React
London, UK
Prices & more details

1
Apr1 week

React Redux Bootcamp
Berlin, Germany
Prices & more details

1
Apr3 days

React Fundamentals
Berlin, Germany
Prices & more details

6
Apr1 day

Advanced React Patterns, FP & Performance
Berlin, Germany
Prices & more details

6
Apr3 days

Advanced React Training
Berlin, Germany
Prices & more details

7
Apr1 day

Advanced React UIs & Styling With Design Systems
Berlin, Germany
Prices & more details

8
Apr1 day

Testing in React
Berlin, Germany
Prices & more details

Using our site means you consent to our use of cookies. Find out more in our privacy policy.