Screenshot of this question was making the rounds last week. But this article covers testing against all the well-known models out there.

Also includes outtakes on the ‘reasoning’ models.

  • ThomasWilliams@lemmy.world
    link
    fedilink
    English
    arrow-up
    3
    arrow-down
    17
    ·
    1 day ago

    <“I want to wash my car. The car wash is 50 meters away. Should I walk or drive?”>

    The model discards the first sentence as it is unrelated to the others.

    Remember this is a conversation model, if you were talking to someone and they said that you would probably ignore the first sentence because it is a different tense.

    • SaltySalamander@fedia.io
      link
      fedilink
      arrow-up
      3
      ·
      17 hours ago

      If I were talking to someone, and said those three sentences, and they chose to ignore the contextual sentence, I would think their social skills were basically nonexistent.

    • Tetragrade@leminal.space
      link
      fedilink
      English
      arrow-up
      8
      arrow-down
      1
      ·
      1 day ago

      Wow you must have done some really extensive probing of the models to say that with such confidence. When can we expect the paper?