Reflections About AI and PuzzleMe

It’s been roughly three months since we launched our AI PuzzleMe Agent, which we described enthusiastically in our monthly newsletter as “the world’s first conversational interface for creating puzzles.”

But our work offering AI tools for puzzle creation began much earlier, almost two years ago. Today about 35% of the hundreds of free puzzles created on our website every day are built using AI, with just a small percentage of those developed using our agent.

Now, with a few months experience with the agent under our belt and much deeper experience with other ways to create puzzles using AI, such as based on topics, PDFs or URLs, we want to talk more deeply about our use of AI.

As you might expect, we have already learned from feedback about our AI Agent by some in the crossword constructor community. We’ve engaged in private conversations and correspondence with some, but we think it’s worthwhile to have a public discussion about AI, too.

We’re grateful that people care about our work enough to give us feedback. We’ve used it to help us plan next steps in our work. (We’ll outline them below as we discuss specific concerns about AI.) We know we’re not always going to agree with our friends in the crossword community. But we hope it’s clear that we value our relationship with professional crossword constructors. We are proud of the work we have done with many to push the boundaries of what’s possible with online puzzles. We see ourselves as part of the crossword community and are committed to helping it and the larger puzzle world flourish.

Before digging into the technical, ethical, philosophical and business issues that have been raised, especially about our AI Agent, it’s important to place our AI work into context.

R&D IS AT THE CORE OF AMUSE LABS

The name of our company, Amuse Labs, reflects our initial ambition, which remains at its core to this day. We see ourselves as an R&D lab working to develop thoughtful education and entertainment. Our focus from the start has been to create new and better online experiences that make word and logic games, and not just crosswords, feel native to the digital world we all live in. To cite just one such example, we introduced social play, which makes it possible for multiple people to solve a puzzle together concurrently. Our goal was to recreate online the wonderful experience of solving puzzles with the help of friends and family.

We have always provided a way for users to create games – whether they are crosswords, quizzes, word searches or others. After the release of ChatGPT three years ago, we saw that an increasing number of users were going to widely available AI tools and asking for suggestions on clues and answers on a particular topic and then pasting them into our puzzle creator form.

While these were not typically professional or expert constructors, we found the newfound ease of creating word games quite interesting and empowering for non-expert individuals.

We also discovered that users were finding our interface a limitation in creating games, given the rather large number of features we have and the variety of people creating such games. For example, people wanted a way to reorder their questions and answers in a quiz, which is not something the interface supported, but which could be easily done with AI. As a result, we started looking at how we could build in some natural language capabilities to make the interface simpler, more intuitive and more flexible.

We’ve focused our use of AI on two public-facing areas. The first is on a help bot to answer user questions 24/7 in real time so people can take advantage of the potential of our software as smoothly as possible. The second is to make it easier and quicker for more people to create puzzles using PuzzleMe.

PuzzleMe Agent combines the existing capabilities of PuzzleMe with the latest LLMs to provide users a simple chat interface to create a variety of puzzles. Our AI Agent assists with the creation of five puzzle types: crosswords, quiz, word search, Word Flower and WordroW.

It’s also important to understand that the AI agent is useful in contexts other than generating puzzle content. For example, it can be used to set options like selecting a backdrop or a completion sound for a puzzle. It can be used to make a puzzle more pleasurable to solve.

CLUES AND PLAGIARISM

One area of confusion and criticism of our AI Agent has been what we’re thought to be doing to train it.

We are not using puzzles created in PuzzleMe or published using PuzzleMe as training data. Our PuzzleMe AI Agent uses existing large language models that have already been trained on internet data by their providers, Open AI and Google. We do not send user puzzles or any PuzzleMe material for model training, and we have never done so.

This brings us to a related important concern: plagiarism. It’s been asserted, for example, that our AI used the clues of Washington Post crossword constructor Evan Birnholz to generate AI clues. We investigated that claim and found that the clue suggestions in question came directly from Google’s Gemini AI model, which is built by scraping public data including Evan’s blog posts written about his puzzles. That said, we agree with the concern around such use, and have embarked on an internal effort to explicitly instruct the AI to try harder to avoid such clues and attribute them better.

We agree that the wholesale adoption of original clues written by others should be discouraged. As we all know, a wealth of crossword clues is readily available to anyone on the web using search or sites like xwordinfo or wordplays. These are commonly used by constructors and editors to research prior usage and avoid repetition. But AI makes the possibility of adopting the work of others even easier. We are working to improve our software to make clear the source of any clue suggestion and remind creators to use suggestions as inspiration, not verbatim, especially if a suggested clue hasn’t been widely used by multiple publications.

QUALITY OF AI GAMES A CONCERN

Another concern we also often hear about the perils of going down the AI game creation path is that AI hallucinates and will introduce factual errors into puzzles. This is related to overall criticism that the use of AI is eroding the quality of puzzles, lowering standards and creating the impression that there’s no need for professional puzzle creators.

It’s important to note that we always remind people using our AI puzzle creator that human oversight, involvement and intervention to ensure accuracy is a necessary part of the process. That isn’t enough for some. But going back to what we said about our roots and goals, we want to help more people to create puzzles that inform and entertain. That’s core to our work.

Despite that, we try to raise red flags along the creation path to make sure people don’t lose sight of the fact that they have a critical role in the quality of a puzzle, even when using powerful AI.

Our documentation about PuzzleMe AI immediately states: “Important: PuzzleMe AI generates content using artificial intelligence. Always review and verify the accuracy of all AI-generated content, including puzzles, clues, questions, and answers, before publishing or relying on it.” Our documentation also states: “PuzzleMe includes AI-powered tools that help creators build puzzles faster and explore new ideas. These features are designed to assist the creative process while keeping authors and editors in control (emphasis added).”

Our first puzzle creation screen states: “Please verify the AI generated content and edit if necessary.”

All that said, we recognize that not everybody will look at our documentation and some may miss the text on the creator screen. We will do more to caution people about the need for verification.

Another general criticism of our AI work is summarized by T Campbell in his New Year’s Day column on our AI: “Even in education, where criss-crosses are more common, an educator has better options to whip up something that’ll adhere to their lesson plan than bringing an unfamiliar AI into the process.”

This is where things get subjective. We don’t believe the fact that others may be doing something should stop us from trying to do something similar even better, as we think we have done in a number of cases. (Teachers have always been allowed to use PuzzleMe for free.) We want PuzzleMe to be a full-service puzzle platform for all types of users and that means giving them the best tools we can for a wide suite of games.

Quality, of course, is always a concern. Some have criticized the fact that our AI Agent isn’t creating dense grid crosswords typical of an American newspaper. These are not the only type of crossword. In fact, about [70-80%] of the thousands of crosswords created on PuzzleMe every month are non-dense grids created by amateur constructors for a variety of use cases where all words focus on a particular theme – family celebrations, educational use, safety training, festivals, or even promoting a TV show. Such users actually embrace having AI support to help them.

This criticism of non-dense grids misses the point that our AI Agent is just the latest iteration of our crossword generation tools, with other approaches we offer making it possible to create dense grids.

AI: A THREAT OR OPPORTUNITY

Many in different lines of work, including creative work, feel their job is threatened by AI. Even our own enterprise is potentially threatened by what LLMs will make possible. We don’t take any such concerns lightly. But we are confident that human creativity and imagination will continue to be what sets apart the best crossword offerings. We as a company have created more opportunities for professional crossword constructors, not reduced the need for them. When asked, we often encourage publishers to invest in hand-crafted crosswords and then identify constructors for them to work with. We will continue to do so.

In the end, the fundamental question about our AI work is why are we doing it. Aren’t we contributing to the demise of rewarding jobs and potentially lowering the quality of puzzles by opening this door?

We hope we’ve shed some light on that question here. The bottom line is we think puzzles and games deserve a much wider place in human life and we’re in the business of trying to make that happen. We’ve seen puzzle creation grow significantly as a result of our AI tools and it’s continuing to grow. We don’t agree that the AI Agent strips “all of the creativity out of building puzzles.” We think it makes it possible for many to tap their own creativity and also at the same time realize the hard work that goes into expert construction.

We’ve heard from some that the time we’re spending on AI is time wasted, taken away from other, more valuable things we could be doing. They say that working to improve the AI agent will lead to a decline in the quality of PuzzleMe. We don’t buy it. AI is an essential part of our work, but it’s not our sole channel of research and development. We have no intention of losing sight of quality. At the same time as we work to develop our AI agent we are taking steps to improve quality, whether the stability and security of our infrastructure or the personal rewards solvers experience when completing a puzzle. We just completed a freewheeling companywide hackathon to surface ideas that could bring satisfaction to new and old users of PuzzleMe alike.

We have learned a lot from the feedback we have already received. We intend to continue to learn and engage with our users and look forward to hearing your response to this blog post. Please write to comments@amuselabs.com.

Thank you.