{"id":11712,"date":"2023-04-01T18:56:05","date_gmt":"2023-04-01T22:56:05","guid":{"rendered":"https:\/\/spinor.info\/weblog\/?p=11712"},"modified":"2023-04-02T02:12:21","modified_gmt":"2023-04-02T06:12:21","slug":"yet-another-revealing-gpt-mistake","status":"publish","type":"post","link":"https:\/\/spinor.info\/weblog\/?p=11712","title":{"rendered":"Yet another revealing GPT mistake"},"content":{"rendered":"

Never mind GPT-4, even GPT-3.5 is amazing. But it has limitations, and studying those limitations can be very educational.<\/p>\n

In this case, I asked it to convert a chessboard, represented using a standard notation<\/a>, into SVG graphics. This was the chessboard string:<\/p>\n

\"rnbqkbnr\/pppp1ppp\/4p3\/8\/4P3\/8\/PPPP1PPP\/RNBQKBNR w KQkq - 0 2\"<\/pre>\n
GPT-3.5 produced sensible SVG code, but the result was less than satisfactory. The chessboard was there, but garbled.<\/p>\n\n\n\n
\n
<svg width=\"320\" height=\"320\">\r\n  <rect x=\"0\" y=\"0\" width=\"320\" height=\"320\" fill=\"#b58863\" \/>\r\n  <rect x=\"0\" y=\"0\" width=\"40\" height=\"40\" fill=\"#f0d9b5\" \/>\r\n  <rect x=\"80\" y=\"0\" width=\"40\" height=\"40\" fill=\"#f0d9b5\" \/>\r\n  <rect x=\"160\" y=\"0\" width=\"40\" height=\"40\" fill=\"#f0d9b5\" \/>\r\n  <rect x=\"240\" y=\"0\" width=\"40\" height=\"40\" fill=\"#f0d9b5\" \/>\r\n  <rect x=\"40\" y=\"40\" width=\"40\" height=\"40\" fill=\"#f0d9b5\" \/>\r\n  <rect x=\"120\" y=\"40\" width=\"40\" height=\"40\" fill=\"#f0d9b5\" \/>\r\n  <rect x=\"200\" y=\"40\" width=\"40\" height=\"40\" fill=\"#f0d9b5\" \/>\r\n  <rect x=\"280\" y=\"40\" width=\"40\" height=\"40\" fill=\"#f0d9b5\" \/>\r\n  <rect x=\"0\" y=\"80\" width=\"40\" height=\"40\" fill=\"#f0d9b5\" \/>\r\n  <rect x=\"80\" y=\"80\" width=\"40\" height=\"40\" fill=\"#b58863\" \/>\r\n  <rect x=\"160\" y=\"80\" width=\"40\" height=\"40\" fill=\"#b58863\" \/>\r\n  <rect x=\"240\" y=\"80\" width=\"40\" height=\"40\" fill=\"#b58863\" \/>\r\n  <rect x=\"40\" y=\"120\" width=\"40\" height=\"40\" fill=\"#b58863\" \/>\r\n  <rect x=\"120\" y=\"120\" width=\"40\" height=\"40\" fill=\"#f0d9b5\" \/>\r\n  <rect x=\"200\" y=\"120\" width=\"40\" height=\"40\" fill=\"#f0d9b5\" \/>\r\n  <rect x=\"280\" y=\"120\" width=\"40\" height=\"40\" fill=\"#f0d9b5\" \/>\r\n  <rect x=\"0\" y=\"160\" width=\"40\" height=\"40\" fill=\"#f0d9b5\" \/>\r\n  <rect x=\"80\" y=\"160\" width=\"40\" height=\"40\" fill=\"#b58863\" \/>\r\n  <rect x=\"160\" y=\"160\" width=\"40\" height=\"40\" fill=\"#b58863\" \/>\r\n  <rect x=\"240\" y=\"160\" width=\"40\" height=\"40\" fill=\"#b58863\" \/>\r\n  <rect x=\"40\" y=\"200\" width=\"40\" height=\"40\" fill=\"#b58863\" \/>\r\n  <rect x=\"120\" y=\"200\" width=\"40\" height=\"40\" fill=\"#f0d9b5\" \/>\r\n  <rect x=\"200\" y=\"200\" width=\"40\" height=\"40\" fill=\"#f0d9b5\" \/>\r\n  <rect x=\"280\" y=\"200\" width=\"40\" height=\"40\" fill=\"#f0d9b5\" \/>\r\n  <rect x=\"0\" y=\"240\" width=\"40\" height=\"40\" fill=\"#f0d9b5\" \/>\r\n  <rect x=\"80\" y=\"240\" width=\"40\" height=\"40\" fill=\"#b58863\" \/>\r\n  <rect x=\"160\" y=\"240\" width=\"40\" height=\"40\" fill=\"#b58863\" \/>\r\n  <rect x=\"240\" y=\"240\" width=\"40\" height=\"40\" fill=\"#b58863\" \/>\r\n  <rect x=\"40\" y=\"280\" width=\"40\" height=\"40\" fill=\"#b58863\" \/>\r\n  <rect x=\"120\" y=\"280\" width=\"40\" height=\"40\" fill=\"#f0d9b5\" \/>\r\n  <rect x=\"200\" y=\"280\" width=\"40\" height=\"40\" fill=\"#f0d9b5\" \/>\r\n  <rect x=\"280\" y=\"280\" width=\"40\" height=\"40\" fill=\"#f0d9b5\" \/>\r\n  <text x=\"50%\" y=\"50%\" fill=\"#000\" font-size=\"50\">\u265a<\/text>\r\n  <text x=\"12.5%\" y=\"12.5%\" fill=\"#000\" font-size=\"50\">\u265c<\/text>\r\n  <text x=\"87.5%\" y=\"12.5%\" fill=\"#000\" font-size=\"50\">\u265c<\/text>\r\n  <text x=\"37.5%\" y=\"12.5%\" fill=\"#000\" font-size=\"50\">\u265e<\/text>\r\n  <text x=\"62.5%\" y=\"12.5%\" fill=\"#000\" font-size=\"50\">\u265e<\/text>\r\n  <text x=\"25%\" y=\"12.5%\" fill=\"#000\" font-size=\"50\">\u265d<\/text>\r\n  <text x=\"75%\" y=\"12.5%\" fill=\"#000\" font-size=\"50\">\u265d<\/text>\r\n  <text x=\"50%\" y=\"87.5%\" fill=\"#fff\" font-size=\"50\">♟<\/text>\r\n  <text x=\"12.5%\" y=\"87.5%\" fill=\"#fff\" font-size=\"50\">♟<\/text>\r\n  <text x=\"87.5%\" y=\"87.5%\" fill=\"#fff\" font-size=\"50\">♟<\/text>\r\n  <text x=\"37.5%\" y=\"87.5%\" fill=\"#fff\" font-size=\"50\">♟<\/text>\r\n  <text x=\"62.5%\" y=\"87.5%\" fill=\"#fff\" font-size=\"50\">♟<\/text>\r\n  <text x=\"25%\" y=\"87.5%\" fill=\"#fff\" font-size=\"50\">♟<\/text>\r\n  <text x=\"75%\" y=\"87.5%\" fill=\"#fff\" font-size=\"50\">♟<\/text>\r\n  <text x=\"50%\" y=\"62.5%\" fill=\"#000\" font-size=\"50\">\u265b<\/text>\r\n<\/svg><\/pre>\n<\/td>\n
\"\"<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
You can almost see how, and where, it went wrong. It started with a dark board in which it then attempted to color lighter squares in a checkerbox pattern, but it lost track in the third row. And when it positioned the pieces, it ended up omitting black’s pawns and white’s non-pawn pieces, while also placing black’s king and queen to where two opposing pawns were supposed to be.<\/p>\n
\"\"<\/p>\n
Even so, this is pretty darn impressive for a language<\/strong> model. Details notwithstanding, it correctly understood the request, and came up with a structure that was not only valid but efficient for the chessboard representation.<\/p>\n
GPT-4 fares a little better but it is still not perfect. It now draws the board correctly but still gets confused about the pawns. Somewhat surprisingly, however, it managed to swap light and dark squares on the board.<\/p>\n\n\n\n
\n
<svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" version=\"1.1\" width=\"320\" height=\"320\" viewBox=\"0 0 320 320\">\r\n  <rect x=\"0\" y=\"0\" width=\"320\" height=\"320\" fill=\"#d18b47\"\/>\r\n  <rect x=\"40\" y=\"0\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <rect x=\"120\" y=\"0\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <rect x=\"200\" y=\"0\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <rect x=\"280\" y=\"0\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <rect x=\"0\" y=\"40\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <rect x=\"80\" y=\"40\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <rect x=\"160\" y=\"40\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <rect x=\"240\" y=\"40\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <rect x=\"40\" y=\"80\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <rect x=\"120\" y=\"80\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <rect x=\"200\" y=\"80\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <rect x=\"280\" y=\"80\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <rect x=\"0\" y=\"120\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <rect x=\"80\" y=\"120\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <rect x=\"160\" y=\"120\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <rect x=\"240\" y=\"120\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <rect x=\"40\" y=\"160\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <rect x=\"120\" y=\"160\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <rect x=\"200\" y=\"160\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <rect x=\"280\" y=\"160\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <rect x=\"0\" y=\"200\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <rect x=\"80\" y=\"200\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <rect x=\"160\" y=\"200\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <rect x=\"240\" y=\"200\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <rect x=\"40\" y=\"240\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <rect x=\"120\" y=\"240\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <rect x=\"200\" y=\"240\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <rect x=\"280\" y=\"240\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <rect x=\"0\" y=\"280\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <rect x=\"80\" y=\"280\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <rect x=\"160\" y=\"280\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <rect x=\"240\" y=\"280\" width=\"40\" height=\"40\" fill=\"#ffce9e\"\/>\r\n  <!-- Chess pieces -->\r\n  <!-- Black pieces -->\r\n  <text x=\"20\" y=\"30\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9820;<\/text>\r\n  <text x=\"60\" y=\"30\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9822;<\/text>\r\n  <text x=\"100\" y=\"30\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9821;<\/text>\r\n  <text x=\"140\" y=\"30\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9819;<\/text>\r\n  <text x=\"180\" y=\"30\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9818;<\/text>\r\n  <text x=\"220\" y=\"30\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9821;<\/text>\r\n  <text x=\"260\" y=\"30\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9822;<\/text>\r\n  <text x=\"300\" y=\"30\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9820;<\/text>\r\n  <text x=\"20\" y=\"70\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9823;<\/text>\r\n  <text x=\"60\" y=\"70\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9823;<\/text>\r\n  <text x=\"100\" y=\"70\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9823;<\/text>\r\n  <text x=\"140\" y=\"70\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9823;<\/text>\r\n  <text x=\"180\" y=\"70\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9823;<\/text>\r\n  <text x=\"220\" y=\"70\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9823;<\/text>\r\n  <text x=\"260\" y=\"70\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9823;<\/text>\r\n  <text x=\"300\" y=\"70\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9823;<\/text>\r\n  <text x=\"180\" y=\"110\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9823;<\/text>\r\n\r\n  <!-- White pieces -->\r\n  <text x=\"20\" y=\"310\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9814;<\/text>\r\n  <text x=\"60\" y=\"310\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9816;<\/text>\r\n  <text x=\"100\" y=\"310\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9815;<\/text>\r\n  <text x=\"140\" y=\"310\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9813;<\/text>\r\n  <text x=\"180\" y=\"310\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9812;<\/text>\r\n  <text x=\"220\" y=\"310\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9815;<\/text>\r\n  <text x=\"260\" y=\"310\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9816;<\/text>\r\n  <text x=\"300\" y=\"310\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9814;<\/text>\r\n  <text x=\"20\" y=\"270\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9817;<\/text>\r\n  <text x=\"60\" y=\"270\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9817;<\/text>\r\n  <text x=\"100\" y=\"270\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9817;<\/text>\r\n  <text x=\"140\" y=\"270\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9817;<\/text>\r\n  <text x=\"220\" y=\"270\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9817;<\/text>\r\n  <text x=\"260\" y=\"270\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9817;<\/text>\r\n  <text x=\"300\" y=\"270\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9817;<\/text>\r\n  <text x=\"100\" y=\"230\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9817;<\/text>\r\n  <text x=\"260\" y=\"230\" font-family=\"Arial\" font-size=\"30\" text-anchor=\"middle\" dy=\".3em\">&#9817;<\/text>\r\n<\/svg><\/pre>\n<\/td>\n
\"\"<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
 <\/p>\n<\/fb:like>","protected":false},"excerpt":{"rendered":"
Never mind GPT-4, even GPT-3.5 is amazing. But it has limitations, and studying those limitations can be very educational. In this case, I asked it to convert a chessboard, represented using a standard notation, into SVG graphics. This was the chessboard string: “rnbqkbnr\/pppp1ppp\/4p3\/8\/4P3\/8\/PPPP1PPP\/RNBQKBNR w KQkq – 0 2” GPT-3.5 produced sensible SVG code, but the […]<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[58],"tags":[],"class_list":["post-11712","post","type-post","status-publish","format-standard","hentry","category-cybernetics","category-58-id","post-seq-1","post-parity-odd","meta-position-corners","fix"],"_links":{"self":[{"href":"https:\/\/spinor.info\/weblog\/index.php?rest_route=\/wp\/v2\/posts\/11712","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/spinor.info\/weblog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/spinor.info\/weblog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/spinor.info\/weblog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/spinor.info\/weblog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=11712"}],"version-history":[{"count":12,"href":"https:\/\/spinor.info\/weblog\/index.php?rest_route=\/wp\/v2\/posts\/11712\/revisions"}],"predecessor-version":[{"id":11728,"href":"https:\/\/spinor.info\/weblog\/index.php?rest_route=\/wp\/v2\/posts\/11712\/revisions\/11728"}],"wp:attachment":[{"href":"https:\/\/spinor.info\/weblog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=11712"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/spinor.info\/weblog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=11712"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/spinor.info\/weblog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=11712"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}