Simon Whaley’s Business of Writing

Testing ProWritingAid’s AI Manuscript Assessment Tool

Welcome to my March 2025 newsletter.

ProWritingAid’s Manuscript Assessment Tool

Those of you who’ve been following my newsletter for a while now will know that I’ve been struggling to finish the third novel in my Mortiforde Mystery series. (Every writer moans about the curse of completing the second novel. Well, all I can say is, just wait until you start the third!)

My first draft was 142,798 words. Not bad, when my target was 90,000 words. (Why 90,000 words? First, that’s generally seen as the maximum length of a cosy crime novel by the market. And, while I self-publish my novels, I would still consider a traditionally published deal—if the right one were to come along—and a traditional publisher would prefer something closer to 90,000 words.)

I’m not a planner, either, so part of that first draft length was me working out what the plot was. I don’t like calling myself a pantster because that’s not a pleasant word. Don’t get wrong, the first draft was pants, but I prefer the phrase ‘discovery writer’ for that’s what I was doing—discovering the story.

My first edit cut that down to 115,589 words, which was a good start, but still a way off the 90,000 target. Edit 2 saw the word count drop to 103,263 words. Better, but still not good. Edit 3 saw me break the 100,000-word barrier—just. After three edits, my manuscript now stood at 99,525. Psychologically, that was a breakthrough.

It was at this point that I received an email from ProWritingAid. As a premium owner of the software, I was entitled to one free manuscript assessment report.

You won’t be surprised to learn that this assessment report uses AI. Now, I don’t like using AI to create text. But, like most writers, I’ve used spelling and grammar checkers for years (Microsoft first started using AI in spelling and grammar checking back in 2016). I was intrigued to know what the AI would come up with.

All I had to do was upload a Word document version of the entire novel and wait for my report. Five minutes later, the AI had read, digested, and assessed all 99,525 words, and generated a report.

Story Overview

The report begins with a general paragraph clarifying the genre and listing the plot outline. This was spot on. It then identified the setting and listed three themes/motifs running through the novel. Again, the AI nailed this.

It then suggested who the novel’s target readers would be, along with listing three similar series of books. Apparently, this third novel is similar to Richard Osman’s Thursday Murder Club series, MC Beaton’s Agatha Raisin series, and Alexander McCall-Smith’s The No. 1 Ladies Detective Agency series. This matches some of the feedback I’ve had from readers and book bloggers.

The report then listed all of my characters, identifying whether they were a protagonist, antagonist, secondary, or tertiary character, and expressing as a percentage the number of times they appear in the novel. Again, this was remarkably accurate.

Table of Contents

The report then broke the novel down into chapters, giving each one a title. I don’t give my chapters a title—but the title it used comprised one or two words summarising the chapter’s events. It used phrases like Missing Funds and Another Body. It then determined the narrative purpose of that chapter, labelling it as an Inciting Incident, Rising Action, Turning Point, Climax, and Resolution. This was really useful for assessing the pace of the book.

Plot and Structure

Next came the assessment report in full, detailing areas that it thought worked well, and others where the AI felt improvement was needed. These are colour-coded as Working Well (green), Minor Concern (yellow), Concern (orange), and Major Concern (red).

I did something good!

In some aspects, this was surprisingly accurate, and in others not. But then, at the moment, that probably sums up all AI at the moment—good when it works, laughable when it hallucinates.

I have to say, I’d had this nagging doubt over one particular element of the novel, which I kept pushing aside and ignoring. Perhaps unsurprisingly, the AI report highlighted this as a Major Concern. Deep down, I knew this was a problem, and here was the AI telling me my gut instinct was right.

What I have found interesting is that the AI has got a lot of the smaller details wrong. and when I say wrong, I mean completely wrong. For example, the report claims that Abigail’s Paris Helicopter Trip was implausible. I would argue it was implausible because Abigail doesn’t go to Paris on a helicopter. Rupert does.

Bit of AI hallucination going on here!

However, even though those finer details were incorrect, it made me review the helicopter trip Rupert undertakes. Time-wise it was plausible, but when I factored in the time of travel (overnight), research revealed that night-time helicopter flights are unusual, particularly for private pilots.

So the detail was wrong, but it still made me review things and consider some further alterations.

Sometimes I completely disagreed with the AI. For example, in one section, it told me that there are no consequences for Lisa accessing a bank account to which she had no authority. This was completely wrong, because in the scene Lisa explains the account she’s accessing is one she DOES legitimately have access to.

Later, it suggested that Abigail’s sudden support for Rosemary is uncharacteristic in Chapter 25. I’ll say so too, because Rosemary doesn’t appear in Chapter 25!

A bit more hallucinating!

And it suggested that a scene concerning a cat toy was unnecessary and then later on exclaimed that a scene that includes a mouse incident is completely unexpected because there was no foreshadowing. (Dear AI, the mouse toy scene is part of that foreshadowing, as were two other scenes throughout the novel.)

Oh, and I was warned that Lisa had divulged another character’s whereabouts in Chapter 10, when this actually happened in Chapter 7.

So, as you can see, some of the finer details were dubious, but overall, I found the bigger picture suggestions extremely useful. At the moment, I feel ProWritingAid’s AI assessment works well at identifying the fundamental issues. The nuanced stuff is best left to humans.

I’ve found this a fascinating exercise, which has been extremely useful in my editing process. I’ve been able to improve the text based on some issues it had flagged up.

This assessment was also a good reminder that I am the human. And as the human, I have the final say. It frustrates me sometimes when I hear some writers applying every grammatical change an AI tool recommends, because AI makes mistakes. That’s not how we should be using AI tools. They are just that: tools. Tools to make us consider whether we need to take action or not. There were some things in the assessment report that I read and then stuck two fingers up at it. Because this is my novel, so I can do what I feel is best. As the author, I have the final say.

I’m happier that I shall pass on a much better manuscript across to my human for checking, which will leave them free to focus on the more nuanced areas.

I’m sure you won’t be surprised to learn that following these amendments, my word count increased to 101,229 words! 😡 But I’m going through yet another edit, and already the text is below 100,500 words, so I’m heading in the right direction again.

Oh, the joys of writing novels!

Bargains

Ooh, Amazon UK are selling my cosy crime, first-in-series, Blooming Murder in paperback format for only £3.50! That's a 65% discount (and cheaper than I can buy it as the author - wish I knew why they are doing that)!

Click here to buy from Amazon

Writing Wednesdays

If you’re looking for more inspiration about writing, check out Matt Lillywhite’s Writing Wednesdays column where he explores the life of being a writer and how to be a better one. He shares some useful advice, and I really agree with his piece on The Real Reason Your Can’t Find Time To Write. We’re all guilty of that one!

Until next month, keeeeeeeeeep writing!

Best wishes,

Simon