Improving The Static Site Created With React And Python

In a previous article I looked at how to create a static site using React and Python. There were a number of issues with the implementation that I wanted to provide better solutions for. You can find the code here. There is a followup article here which demonstrates the use of Web Workers to run the python code in a separate thread.

Global Variables Suck

The first issue was with how I was calling the Python code. My code looked like this:

export function dotProductExerciseCode(maxNumberOfColumns, maxNumberOfRows) {
    return `
import random
import numpy as np
m = random.randint(1, ${maxNumberOfRows})
n = random.randint(1, ${maxNumberOfColumns})
p = random.randint(1, ${maxNumberOfColumns})
rng = np.random.default_rng()
A = rng.integers(low=-10, high=10, size=(m, n), dtype=np.int32)
B = rng.integers(low=-10, high=10, size=(n, p), dtype=np.int32)
C = A @ B
`
  }

As you can see the JavaScript function inserted the maximum rows and columns, and the results were left in global variables. Clearly this looks like it should be a function. Here’s what I came up with.

import random
import numpy as np
# Make the function async so it becomes a promise in JavaScript.
async def generate_matmul_exercise(max_rows, max_cols):
    m = random.randint(1, max_rows)
    n = random.randint(1, max_cols)
    p = random.randint(1, max_cols)
    rng = np.random.default_rng()
    A = rng.integers(low=-10, high=10, size=(m, n), dtype=np.int32)
    B = rng.integers(low=-10, high=10, size=(n, p), dtype=np.int32)
    C = A @ B
    return {
        'm': m,
        'n': n,
        'p': p,
        'A': A,
        'B': B,
        'C': C
    }
async def generate_matadd_exercise(max_rows, max_cols):
    m = random.randint(1, max_rows)
    n = random.randint(1, max_cols)
    rng = np.random.default_rng()
    A = rng.integers(low=-10, high=10, size=(m, n), dtype=np.int32)
    B = rng.integers(low=-10, high=10, size=(m, n), dtype=np.int32)
    C = A + B
    return {
        'm': m,
        'n': n,
        'A': A,
        'B': B,
        'C': C
    }

Much of this is obvious. The functions take arguments, and return the result in a dictionary. There are two points to note.

The functions are async. This means when I call them from JavaScript they return a promise, so if the function takes a long time to execute it won’t block the UI.

Also there are two functions! The previous approach wasn’t modular and would not scale well. I think this solution seems clearer, more natural, and more “Pythonic”.

Calling Functions

Now I want to call the functions from JavaScript in a more natural way. As before the output arguments need transforming. This is what I came up with:

function wrapGenerateMatmulExercise(pyodide) {
  const func = pyodide.globals.get('generate_matmul_exercise')
  return async (maxNumberOfRows, maxNumberOfColumns) => {
    const obj = await func(maxNumberOfRows, maxNumberOfColumns)
    const results = {
      m: obj.get('m'),
      n: obj.get('n'),
      p: obj.get('p'),
      A: obj
        .get('A')
        .toJs()
        .map((x) => Array.from(x)),
      B: obj
        .get('B')
        .toJs()
        .map((x) => Array.from(x)),
      C: obj
        .get('C')
        .toJs()
        .map((x) => Array.from(x))
    }
    obj.destroy()
    return results
  }
}

I start by getting the function from the pyodide.globals. Then I return a wrapper function which handles the transformation of the outputs.

Note the obj.destroy() near the end. This was another issue with the previous solution. Where the browser supports it pyodide will wrap the objects returned from Python in “finalizers”. This means the memory allocated in the Python wasm memory will get freed eventually. The problem here is the JavaScript engine is unaware of the memory used in the wasm part, and the JavaScript proxy objects returned are very small. This means the finalizers may never be called! This would be a major issue if the Python code was holding a large amount of data.

Scalability

The final issue I wanted to address was scalability. I’ve already gone some way to addressing this by defining functions rather than using global variables. However the following adds a little more organisation:

export async function importLocalPythonCode(pyodide) {
  await pyodide.runPythonAsync(PYTHON_CODE)
  const generateMatmulExercise = wrapGenerateMatmulExercise(pyodide)
  const generateMataddExercise = wrapGenerateMataddExercise(pyodide)
  return {
    generateMatmulExercise,
    generateMataddExercise
  }
}

The initialisation of the Python environment now looks like this:

async componentDidMount() {
    const pyodide = await window.loadPyodide({
      indexURL: 'https://cdn.jsdelivr.net/pyodide/v0.17.0/full/'
    })
    const localPythonCode = await importLocalPythonCode(pyodide)
    this.setState({ localPythonCode })
  }

This looks much neater to my eye, and feels like a good starter template for larger projects. You can find the code here.