Isomorphic / universal React at Gousto

About half a year ago, we decided to build our new frontend stack using isomorphic/universal JavaScript leveraging React and its rich ecosystem both on client and server side.

Initially, we started out using React to create single page applications (SPA). This worked quite well, we had a clean and scalable JavaScript codebase removing the need for jQuery and other libraries.

However, as with any traditional SPA, you lose server side rendering. This results in a slow website and disappointing user experience. With an SPA the server usually renders a minimal HTML page with a single <div/>, script and css files, then the browser needs to evaluate JavaScript, bootstrap the application and start fetching data. Once the data is finally returned the UI can be rendered. The problem with this model is the chained nature of loading resources: the initial page, render blocking javascript and css then data through ajax. Not to mention, the lost SEO value for public facing pages (even though Google is evaluating JS, SPAs tend to have worse rankings).

One of the most interesting solutions to these problems is isomorphic (or recently called universal) JavaScript.

The application

The main features and goal for the application are:

  • Server side rendering: render meaningful pages by fetching data on the server which can then also be used on the client, with no extra ajax calls.
  • Routing: be able to handle routing client and server side. Leveraging client side routing between pages so no full page reload is needed, while having the ability for any page to be served form the server with fully generated HTML markup
  • State management: sharing server side application state with the client to decrease UI flashing and decrease the number of ajax calls
  • Code reuse: be able to use the same code on the client and server side to decrease development time and reduce bugs

The main components of our isomorphic stack are:

We managed to share over 95% of our JavaScript codebase between the server and client, thanks to libraries which can work on both client and server side.

Koa

At Gousto we use Koa (version 2) with node as it is extremely simple to use and more performant than Express.

Routing

React router can be used on the client and on the server. This allows us to use the exact same routes definition for client and server, however server side rendering requires match rather than <Router>.

Note, that the example code snippets below only show the main concepts, and leave out import statements and such.

example code for server/main.js which handles the :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
const app = new Koa()

app.use(async (ctx, next) => {
await new Promise((resolve, reject) => {
// `history` is memoryHistory, `store` is redux store, `routes` is usual react-router routes
match({ history, routes, location: ctx.request.url }, (error, redirectLocation, renderProps) => {
if (renderProps) {
ctx.body = htmlTemplate(renderHTML(store, renderProps), store.getState()) // see below
resolve()
}
})
})
await next()
})

app.listen(3000)

React

React supports server side rendering through ReactDOMServer.renderToString. We can use renderToString to generate the HTML markup on the server. Data is specified via props, same as on the client.
Once React is running on the client it will render itself to the DOM container specified. In addition, as long as the server generated output is the same as the client’s, React does not re-render the content of this DOM element, instead, just adding event listeners and running client specific lifecycle events.

This requires the exact same data to be available on the client as what was used on the server.

Redux

We use Redux with ImmutableJS for state management. If redux is initialised with no data, react will render an empty page on the client and would lose the server rendered output, with a huge flash on the page.
To provide data to client from the server, the redux store can be serialised and dumped into the HTML response in a script tag, to then be used by React as initial state for the Redux store on client side.
For stores using ImmutableJS, transit-immutable-js provides an easy way to serialise server side application state.

example code for server/template.js:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
const htmlTemplate = (reactHTML = '', initialState = {}) => (
`<!doctype html>
<html lang="en-us">
<head>
<script type="text/javascript">
window.__initialState__ = ${JSON.stringify(transit.toJSON(initialState))} // let the client know the server-side state
</script>
</head>
<body>
<script src="client.bundle.js" async></script> // include the client bundle
<div id="react-root">${reactHTML}</div> // use the server rendered markup
</body>
</html>`
)

addition to server/app.js:

1
2
3
4
5
6
7
8
9
10
11
12
13
const renderHTML = (store, renderProps) => {
const components = (
<Provider store={store}> // use redux
<RouterContext {...renderProps} /> // the main app through `renderProps`, matched components with react router
</Provider>
)
const reactHTML = ReactDOM.renderToString(components)

return reactHTML
}

const app = new Koa()
... route matching .... (see example above)

example code for client.js:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
let initialState = window.__initialState__ || '{}'
initialState = transit.fromJSON(initialState)
const store = configureStore(browserHistory, initialState) // configureStore adds redux reducers, middlewares and actions

const reactRootDOM = document.getElementById('react-root')

if (reactRootDOM) {
ReactDOM.render(
<Provider store={store}> // redux
<GoustoReactApp /> // main app which includes routing
</Provider>,
reactRootDOM
)
}

This in essence will create a basic universal JavaScript application which can render the same page client and server side.

Fetching data & sharing state

Client side data fetching is pretty easy as many of us are familiar with it. You can use callbacks or promises, everything happens async, once the client receives the response the callback is called or the promise gets resolved. The UI is constantly updated and re-rendered automatically.

Server side data fetching works pretty much the same, however, the UI does not get automatically and constantly updated. On the server the components are only rendered once, after react router matches the current path.
As rendering only happens once on the server, react lifecycle hooks for data fetching cannot be used.
All data needs to be available before ReactDOM.renderToString runs.
This means that before rendering, only the components matched by react router are known, their child components are not.

By adding static fetchData methods to components which are mapped to routes, data can be fetched before the response is generated.
For example: gousto/menu uses Menu.js which uses many different other components such as <Recipe>, <BoxSummary> etc.
Recipe and BoxSummary are not known before ReactDOM.renderToString runs, so we need to use the fetchData methods from the matched components.

example code for routes.js:

1
2
3
4
5
<Route path="/" component={MainLayout}>
<IndexRoute component={Home} />
<Route path="/menu(/:orderId)" component={Menu} />
<Route path="/my-account" component={MyAccount} />
</Route>

example code for Menu.js:

1
2
3
4
5
6
7
8
9
class Menu extends React.PureComponent {
static fetchData({ store, query, params }) {
return store.dispatch(actions.loadMenu()) // which will return a promise
}

render() {
...
}
}

In this example, if /menu is loaded, fetchData from Menu.js and from MainLayout.js can be used.
These methods need to be called and their response handled before the server returns the response.

server/main.js should be modified to handle data fetching:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
const fetchAllData = (renderProps, store) => {
const { location, params } = renderProps
const queries = []

renderProps.components.forEach(component => {
if (component) {
let fetchData
if (typeof component.fetchData === 'function') {
fetchData = component.fetchData
}

if (fetchData) {
queries.push(
new Promise((resolve) => {
resolve(fetchData({ params, query: location.query, store }))
})
)
}
}
})

return Promise.all(queries) // run fetches in parallel
}

app.use(async (ctx, next) => {
await new Promise((resolve, reject) => {
match({ history, routes, location: ctx.request.url }, async (error, redirectLocation, renderProps) => {
if (renderProps) {
await fetchAllData(renderProps, store) // wait for all the data to be available
ctx.body = htmlTemplate(renderHTML(store, renderProps), store.getState())
resolve()
}
})
})
await next()
})

In this example if a user visits /menu, fetchAllData will call both Menu.fetchData and MainLayout.fetchData parallel, thanks to Promise.all.
This will populate the redux store with data needed to render the page (in this case recipes & stock for the latest menu).
Menu.fetchData will not be called on the client side as the HTML is already built on the server and all the data is inserted into the response, available through window.__initialState__ for the client to use it when initialising the redux store. (see above for client code example)

At Gousto, for certain pages we actually need quite a lot of data which can come from many different microservices. Calling these API endpoints mean we can drastically slow down the site. To mitigate this, within every <Component>.fetchData we parallelise data fetching. .fetchDatas are also run in parallel by fetchAllData (see above), ie we try to return a promise which fetches data in parallel and then all the different promises from several .fetchDatas run in parallel too.

This actually means we can add many API calls to every page load, as the total time used for data fetching equals the time for the slowest call - provided we do not chain any calls. This has a nice side effect, as long as a call is faster than a previous and can be called parallel, it will not slow the site down at all.

API calls which are too slow and would slow down the whole website can be taken out from the server and moved onto the client, provided the UI will not flicker too much or the data is not instantly needed.
For us at Gousto, one example is the box prices call. This call will take around 100ms (or more) if a user has an active discount. this will slow down the website significantly, even when everything else is parallelised.

On the menu, to display the box details panel, the user needs to click on the box summary button which will bring up box details. This panel includes the pricing information. By default, this box is in closed state. This means, pricing information can be moved from loading on the server to the client as it is not needed for the server output. It is highly likely that by the time the user clicks on the box summary button the data has been successfully loaded on the client and can be used.

React has the special componentDidMount lifecycle method which will be called only when running in the browser. This helped us to split loading data between client and server to optimise speed.
Another way to optimise loading data is using the redux store to check what data is already available for the client and load the rest when needed instead of reloading all data in componentDidMount on every client side navigation.

updated code for Menu.js:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
class Menu extends React.PureComponent {
static fetchData({ store, query, params }) {
return store.dispatch(actions.loadMenu())
}

componentDidMount() {
this.props.loadStock()
if (this.props.boxPrices.size === 0) {
this.props.loadBoxPrices()
}
if (this.props.menu.size === 0) {
this.props.loadMenu()
}
}

render() {
...
}
}

In Menu.js the recipes on the menu are only fetched once while the application is running. This could come from the server or any other page as the application state is shared between pages. If a visitor lands on a page which loads the current menu, when navigating to /menu the application will not have to fetch the entire menu again. However stock will be fetched every time a user either lands on this page, or comes through client side routing as stock is more volatile.

Build

Currently, Webpack is probably the most advanced and widely adopted build tool. It can bundle files or run tasks such as minification.
An isomorphic/universal application may require two builds, probably one for the server (to be able to require css or images) and certainly another one for the client.
To leverage the latest JavaScript advancements we use Babel for code transpiling. This allows us to use ES6 and ES7 features.
For CSS, we prefer CSS Modules to create local-only classes and shrinking the css bundle to a minimum.
Using webpack, for server and client, we can require/import css files and even images or fonts.

Conclusion

Swapping to universal frontend has decreased bugs, increased site speed and percieved performance against client-only SPAs and helps us move faster compared to server rendered HTML with jQuery.
Our focus for the future is to make all our pages powered by the new stack, to leverage client and server side routing with data fetching and data sharing between different pages.

Balint
Senior Frontend Engineer