Adventures with ChatGPT 4 – Writing the Perfect Case Study

I was holding back on upgrading my ChatGPT account to a paid version until this past week. A podcast contrasting GPT 3.5 to GPT 4 was the tipping point. Vast improvements lie ahead. For me, improved AI-assisted writing was the place to start. It was an easy place to assess how creative it could be, how close to emulating human communication it could come.

At Point of Reference, we don’t produce the conventional one-page customer case studies. Or, as ChatGPT would describe these pieces: “meticulously curated and professionally enhanced customer case studies, optimized for brand alignment and audience engagement.” It turns out that our buyers, customer marketers, rarely downloaded case studies from our website. Go figure! We make more in-depth, organic (a.k.a. unvarnished) customer interviews available on our website on the Our Advocates page. One of these interviews, featuring Colette Chavalia of Databricks, was used as the source content for GPT 4 to do its magic. Here’s the process I followed to get to the end result:

  • My first (of many) prompts was to “write a 500-word customer case study using this interview transcript.” For comparison, you can see the original transcript, as well as the finished product (no formatting, just copy).
  • I learned, when working with different interview transcripts, that GPT 4 has a prompt limit of 2,048 characters. If your transcripts are longer than that, just start with the first 2,048 characters, and regenerate a response.
  • You can copy the next 2,048 characters and tell it to “incorporate this additional copy into the case study.” Then repeat as often as needed.
  • Not only will the additional content result in different versions, the Regenerate button will produce different flavors of the same information. You could literally do this all day long since there’s something you’ll like about one version, but not another. Same goes for specific things you don’t like.
  • Some of the word choices used to summarize benefits and challenges were quite good, and different from what I would have come up with. Loved that.
  • Each iteration highlighted different aspects of the interview and even used different headings and groupings. So I picked what I wanted and gave it prompts with specific inclusions and exclusions. It effortlessly kept track of it’s history, which was impressive. A “longer memory” is one of the key improvements in GPT 4.
  • After a few rounds, I realized that GPT hadn’t used many customer quotes, which in our opinion, are the best, most authentic part of any customer story. So, I prompted it to “include as many compelling quotes as possible, “while keeping to the word count limit of 550.” I gave it a bit more space to work with.
  • Ultimately, I got something that was 90% to my personal and highly subjective satisfaction. There were still some parts that weren’t quite right, but it brought me to a good starting point for my final editing process.

Did it save time? Actually, it took longer than I expected to get to that “last mile” version, but I’m sure it gets faster as you learn how best to interact with ChatGPT for specific needs. One thing I didn’t try this time around was to prompt it to emulate a writing style. This could be based on other similar documents written in a desirable way, or some well-known author, or actor for that matter. The possibilities are endless.

What’s really important, at least to me, is that AI-supported copy doesn’t become generic, predictable business-speak. That just won’t achieve the level of engagement that unexpected, interesting, thought or emotion-provoking copy can. Of course, this can happen if there is no AI involved at all, so keep that in mind when comparing the “authors.”

To get to a happy place with GPT 4, it needs a fair amount of human guidance. Definitely not a one-and-done affair. And keep in mind, this is THE most mature generative AI technology available. And for only $20 per month. That’s mind-boggling.

Over time I could envision making a list of likes and dislikes, guidelines for consistency, and including that in a prompt used early in the copy generating process. That would save time by eliminating some of the iterations. The leap from the 3.5 to 4 model was, I heard on that podcast, like going from a bee to a squirrel’s brain size. Not sure exactly how that comparison is made, but it seems substantial on the surface. Makes me wonder what surprises are ahead for version 5, 6 and beyond. When will the brain be bigger than a human’s, and what brain is that!?

I hope this gives you some ideas, whether you’re still on the fence, or already using AI for writing. If you’ve been using GPT 4 for this purpose, what’s your experience been? Any tips for maximizing the efficiency of the process and getting what you consider high quality?