No, I am not worried about being eaten by a grue in the dark, as in the Great Underground Empire of the classic Zork text adventure games (if you ever played those games, you cannot possibly forget the ominous warning: “It is pitch black. You are likely to be eaten by a grue.”) Nor am I a secret admirer of Glavnoye razvedyvatel’noye upravleniye, the former USSR’s intelligence directorate, or its Putinist successor institution.
Rather, I am talking about networks of gated recurrent units, a machine learning architecture that is well suited to analyze time series data. I’ve been using “black box” GRU implementations for some time in a research project, but it’s one thing to learn to use a software library, it’s another thing to understand the conceptual details.
It is for that reason that (with the help of our sophisticated LLM friends) I embarked on a side project of building my own GRU network, in plain C++ code, without relying on other people’s solutions. That’s the best way to understand a software solution: Build your own!
Which may explain why I get excited when I manage to produce a plot like this:
Nothing fancy, just an amplitude-modulated carrier (red), with a lower frequency modulating signal (green).
But here’s the point: The GRU network doesn’t know a thing about amplitude modulation. It just learns the relationship between red and green. And learn it does: after a few passes using a training data set, it manages to reproduce the modulating signal with decent accuracy.
My code likely still contains subtle errors, as I suspect that it can do even better. A lot also depends on the model “hyperparameters”, parameters that define the model and control the training process. Even so, I am pleased and excited: It is so much fun, seeing a creation like this “come to life”, working as it is supposed to, doing some nontrivial software magic in a mere, what, maybe 700 lines of code, but that actually even includes some commented-out lines.