Australian government trial finds AI is much worse than humans at summarizing
arstechnica.com
Llama2-70B failed to capture "complex context," but updated models might do better.
  • Python@programming.devEnglish
    201·
    2 months ago

    I find AI summaries so damn tedious to read, they just keep repeating themselves over and over again :(

        • pirat@lemmy.worldEnglish
          7·
          2 months ago

          In conclusion, the conclusion most often concludes by concluding that we can conclude, that the answer is the answer.

          To summarize, the summary most often summarizes the problem and its nuances, and concludes by summarizing the conclusion as the correct answer.

          Therefore, the correct answer is correct.

          Let me know if there is anything else I can help you with today.

          • P1nkman@lemmy.worldEnglish
            3·
            2 months ago

            And if it does not work, let’s try another approach.

            Gives exact same information/instructions

  • hendrik@palaver.p3x.deEnglish
    5·
    2 months ago

    Lots of people seem to like them. I’ve read several AI summaries also here on Lemmy. And they kind of always miss the nuance. Oftentimes they entirely miss the point. So much that it sometimes becomes wrong.

  • Ilovethebomb@lemm.eeEnglish
    41·
    2 months ago

    One evaluator highlighted this problem by calling out an AI summary for being “wordy and pointless—just repeating what was in the submission.”

    Basically sums up my view of AI generated text, it has a certain waffle to it that is very distinctive, regardless of the source.

  • daddy32@lemmy.worldEnglish
    21·
    2 months ago

    Clickbait.

    They tested it on a single, year old model (Llama2-70B), which is way below state of the art.