{"id":12522,"date":"2024-03-26T03:28:02","date_gmt":"2024-03-26T07:28:02","guid":{"rendered":"https:\/\/spinor.info\/weblog\/?p=12522"},"modified":"2024-03-26T03:28:02","modified_gmt":"2024-03-26T07:28:02","slug":"gru","status":"publish","type":"post","link":"https:\/\/spinor.info\/weblog\/?p=12522","title":{"rendered":"GRU"},"content":{"rendered":"

No, I am not worried about being eaten by a grue in the dark, as in the Great Underground Empire of the classic Zork text adventure games (if you ever played those games, you cannot possibly forget the ominous warning: “It is pitch black. You are likely to be eaten by a grue.”) Nor am I a secret admirer of Glavnoye razvedyvatel’noye upravleniye<\/i>, the former USSR’s intelligence directorate, or its Putinist successor institution.<\/p>\n

Rather, I am talking about networks of gated recurrent units, a machine learning architecture that is well suited to analyze time series data. I’ve been using “black box” GRU implementations for some time in a research project, but it’s one thing to learn to use a software library, it’s another thing to understand the conceptual details.<\/p>\n

It is for that reason that (with the help of our sophisticated LLM friends) I embarked on a side project of building my own GRU network, in plain C++ code, without relying on other people’s solutions. That’s the best way to understand a software solution: Build your own!<\/p>\n

Which may explain why I get excited when I manage to produce a plot like this:<\/p>\n

\"\"<\/p>\n

Nothing fancy, just an amplitude-modulated carrier (red), with a lower frequency modulating signal (green).<\/p>\n

But here’s the point: The GRU network doesn’t know a thing about amplitude modulation. It just learns the relationship between red and green. And learn it does: after a few passes using a training data set, it manages to reproduce the modulating signal with decent accuracy.<\/p>\n

My code likely still contains subtle errors, as I suspect that it can do even better. A lot also depends on the model “hyperparameters”, parameters that define the model and control the training process. Even so, I am pleased and excited: It is so much fun, seeing a creation like this “come to life”, working as it is supposed to, doing some nontrivial software magic in a mere, what, maybe 700 lines of code, but that actually even includes some commented-out lines.<\/p>\n<\/fb:like>","protected":false},"excerpt":{"rendered":"

No, I am not worried about being eaten by a grue in the dark, as in the Great Underground Empire of the classic Zork text adventure games (if you ever played those games, you cannot possibly forget the ominous warning: “It is pitch black. You are likely to be eaten by a grue.”) Nor am […]<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[58],"tags":[],"class_list":["post-12522","post","type-post","status-publish","format-standard","hentry","category-cybernetics","category-58-id","post-seq-1","post-parity-odd","meta-position-corners","fix"],"_links":{"self":[{"href":"https:\/\/spinor.info\/weblog\/index.php?rest_route=\/wp\/v2\/posts\/12522","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/spinor.info\/weblog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/spinor.info\/weblog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/spinor.info\/weblog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/spinor.info\/weblog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=12522"}],"version-history":[{"count":2,"href":"https:\/\/spinor.info\/weblog\/index.php?rest_route=\/wp\/v2\/posts\/12522\/revisions"}],"predecessor-version":[{"id":12525,"href":"https:\/\/spinor.info\/weblog\/index.php?rest_route=\/wp\/v2\/posts\/12522\/revisions\/12525"}],"wp:attachment":[{"href":"https:\/\/spinor.info\/weblog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=12522"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/spinor.info\/weblog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=12522"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/spinor.info\/weblog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=12522"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}