Yet another revealing GPT mistake

Apr 012023

Never mind GPT-4, even GPT-3.5 is amazing. But it has limitations, and studying those limitations can be very educational.

In this case, I asked it to convert a chessboard, represented using a standard notation, into SVG graphics. This was the chessboard string:

"rnbqkbnr/pppp1ppp/4p3/8/4P3/8/PPPP1PPP/RNBQKBNR w KQkq - 0 2"

GPT-3.5 produced sensible SVG code, but the result was less than satisfactory. The chessboard was there, but garbled.

<svg width="320" height="320">
  <rect x="0" y="0" width="320" height="320" fill="#b58863" />
  <rect x="0" y="0" width="40" height="40" fill="#f0d9b5" />
  <rect x="80" y="0" width="40" height="40" fill="#f0d9b5" />
  <rect x="160" y="0" width="40" height="40" fill="#f0d9b5" />
  <rect x="240" y="0" width="40" height="40" fill="#f0d9b5" />
  <rect x="40" y="40" width="40" height="40" fill="#f0d9b5" />
  <rect x="120" y="40" width="40" height="40" fill="#f0d9b5" />
  <rect x="200" y="40" width="40" height="40" fill="#f0d9b5" />
  <rect x="280" y="40" width="40" height="40" fill="#f0d9b5" />
  <rect x="0" y="80" width="40" height="40" fill="#f0d9b5" />
  <rect x="80" y="80" width="40" height="40" fill="#b58863" />
  <rect x="160" y="80" width="40" height="40" fill="#b58863" />
  <rect x="240" y="80" width="40" height="40" fill="#b58863" />
  <rect x="40" y="120" width="40" height="40" fill="#b58863" />
  <rect x="120" y="120" width="40" height="40" fill="#f0d9b5" />
  <rect x="200" y="120" width="40" height="40" fill="#f0d9b5" />
  <rect x="280" y="120" width="40" height="40" fill="#f0d9b5" />
  <rect x="0" y="160" width="40" height="40" fill="#f0d9b5" />
  <rect x="80" y="160" width="40" height="40" fill="#b58863" />
  <rect x="160" y="160" width="40" height="40" fill="#b58863" />
  <rect x="240" y="160" width="40" height="40" fill="#b58863" />
  <rect x="40" y="200" width="40" height="40" fill="#b58863" />
  <rect x="120" y="200" width="40" height="40" fill="#f0d9b5" />
  <rect x="200" y="200" width="40" height="40" fill="#f0d9b5" />
  <rect x="280" y="200" width="40" height="40" fill="#f0d9b5" />
  <rect x="0" y="240" width="40" height="40" fill="#f0d9b5" />
  <rect x="80" y="240" width="40" height="40" fill="#b58863" />
  <rect x="160" y="240" width="40" height="40" fill="#b58863" />
  <rect x="240" y="240" width="40" height="40" fill="#b58863" />
  <rect x="40" y="280" width="40" height="40" fill="#b58863" />
  <rect x="120" y="280" width="40" height="40" fill="#f0d9b5" />
  <rect x="200" y="280" width="40" height="40" fill="#f0d9b5" />
  <rect x="280" y="280" width="40" height="40" fill="#f0d9b5" />
  <text x="50%" y="50%" fill="#000" font-size="50">♚</text>
  <text x="12.5%" y="12.5%" fill="#000" font-size="50">♜</text>
  <text x="87.5%" y="12.5%" fill="#000" font-size="50">♜</text>
  <text x="37.5%" y="12.5%" fill="#000" font-size="50">♞</text>
  <text x="62.5%" y="12.5%" fill="#000" font-size="50">♞</text>
  <text x="25%" y="12.5%" fill="#000" font-size="50">♝</text>
  <text x="75%" y="12.5%" fill="#000" font-size="50">♝</text>
  <text x="50%" y="87.5%" fill="#fff" font-size="50">♟</text>
  <text x="12.5%" y="87.5%" fill="#fff" font-size="50">♟</text>
  <text x="87.5%" y="87.5%" fill="#fff" font-size="50">♟</text>
  <text x="37.5%" y="87.5%" fill="#fff" font-size="50">♟</text>
  <text x="62.5%" y="87.5%" fill="#fff" font-size="50">♟</text>
  <text x="25%" y="87.5%" fill="#fff" font-size="50">♟</text>
  <text x="75%" y="87.5%" fill="#fff" font-size="50">♟</text>
  <text x="50%" y="62.5%" fill="#000" font-size="50">♛</text>
</svg>

You can almost see how, and where, it went wrong. It started with a dark board in which it then attempted to color lighter squares in a checkerbox pattern, but it lost track in the third row. And when it positioned the pieces, it ended up omitting black’s pawns and white’s non-pawn pieces, while also placing black’s king and queen to where two opposing pawns were supposed to be.

Even so, this is pretty darn impressive for a language model. Details notwithstanding, it correctly understood the request, and came up with a structure that was not only valid but efficient for the chessboard representation.

GPT-4 fares a little better but it is still not perfect. It now draws the board correctly but still gets confused about the pawns. Somewhat surprisingly, however, it managed to swap light and dark squares on the board.

<svg xmlns="http://www.w3.org/2000/svg" version="1.1" width="320" height="320" viewBox="0 0 320 320">
  <rect x="0" y="0" width="320" height="320" fill="#d18b47"/>
  <rect x="40" y="0" width="40" height="40" fill="#ffce9e"/>
  <rect x="120" y="0" width="40" height="40" fill="#ffce9e"/>
  <rect x="200" y="0" width="40" height="40" fill="#ffce9e"/>
  <rect x="280" y="0" width="40" height="40" fill="#ffce9e"/>
  <rect x="0" y="40" width="40" height="40" fill="#ffce9e"/>
  <rect x="80" y="40" width="40" height="40" fill="#ffce9e"/>
  <rect x="160" y="40" width="40" height="40" fill="#ffce9e"/>
  <rect x="240" y="40" width="40" height="40" fill="#ffce9e"/>
  <rect x="40" y="80" width="40" height="40" fill="#ffce9e"/>
  <rect x="120" y="80" width="40" height="40" fill="#ffce9e"/>
  <rect x="200" y="80" width="40" height="40" fill="#ffce9e"/>
  <rect x="280" y="80" width="40" height="40" fill="#ffce9e"/>
  <rect x="0" y="120" width="40" height="40" fill="#ffce9e"/>
  <rect x="80" y="120" width="40" height="40" fill="#ffce9e"/>
  <rect x="160" y="120" width="40" height="40" fill="#ffce9e"/>
  <rect x="240" y="120" width="40" height="40" fill="#ffce9e"/>
  <rect x="40" y="160" width="40" height="40" fill="#ffce9e"/>
  <rect x="120" y="160" width="40" height="40" fill="#ffce9e"/>
  <rect x="200" y="160" width="40" height="40" fill="#ffce9e"/>
  <rect x="280" y="160" width="40" height="40" fill="#ffce9e"/>
  <rect x="0" y="200" width="40" height="40" fill="#ffce9e"/>
  <rect x="80" y="200" width="40" height="40" fill="#ffce9e"/>
  <rect x="160" y="200" width="40" height="40" fill="#ffce9e"/>
  <rect x="240" y="200" width="40" height="40" fill="#ffce9e"/>
  <rect x="40" y="240" width="40" height="40" fill="#ffce9e"/>
  <rect x="120" y="240" width="40" height="40" fill="#ffce9e"/>
  <rect x="200" y="240" width="40" height="40" fill="#ffce9e"/>
  <rect x="280" y="240" width="40" height="40" fill="#ffce9e"/>
  <rect x="0" y="280" width="40" height="40" fill="#ffce9e"/>
  <rect x="80" y="280" width="40" height="40" fill="#ffce9e"/>
  <rect x="160" y="280" width="40" height="40" fill="#ffce9e"/>
  <rect x="240" y="280" width="40" height="40" fill="#ffce9e"/>
  <!-- Chess pieces -->
  <!-- Black pieces -->
  <text x="20" y="30" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9820;</text>
  <text x="60" y="30" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9822;</text>
  <text x="100" y="30" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9821;</text>
  <text x="140" y="30" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9819;</text>
  <text x="180" y="30" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9818;</text>
  <text x="220" y="30" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9821;</text>
  <text x="260" y="30" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9822;</text>
  <text x="300" y="30" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9820;</text>
  <text x="20" y="70" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9823;</text>
  <text x="60" y="70" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9823;</text>
  <text x="100" y="70" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9823;</text>
  <text x="140" y="70" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9823;</text>
  <text x="180" y="70" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9823;</text>
  <text x="220" y="70" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9823;</text>
  <text x="260" y="70" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9823;</text>
  <text x="300" y="70" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9823;</text>
  <text x="180" y="110" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9823;</text>

  <!-- White pieces -->
  <text x="20" y="310" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9814;</text>
  <text x="60" y="310" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9816;</text>
  <text x="100" y="310" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9815;</text>
  <text x="140" y="310" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9813;</text>
  <text x="180" y="310" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9812;</text>
  <text x="220" y="310" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9815;</text>
  <text x="260" y="310" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9816;</text>
  <text x="300" y="310" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9814;</text>
  <text x="20" y="270" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9817;</text>
  <text x="60" y="270" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9817;</text>
  <text x="100" y="270" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9817;</text>
  <text x="140" y="270" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9817;</text>
  <text x="220" y="270" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9817;</text>
  <text x="260" y="270" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9817;</text>
  <text x="300" y="270" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9817;</text>
  <text x="100" y="230" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9817;</text>
  <text x="260" y="230" font-family="Arial" font-size="30" text-anchor="middle" dy=".3em">&#9817;</text>
</svg>

You must be logged in to post a comment.

Spinor Info

Yet another revealing GPT mistake

Leave a Reply