A few conversations with a chatbot *

CBS Television., Public domain, via Wikimedia Commons

If you've started using the most-common, readily available AI chatbots/Assistants (e.g., ChatGPT, Google Gemini, Claude, Microsoft Copilot) you've experienced the speed at which they can consolidate and tabulate information. I recently spent some time using Google Gemini to explore some concepts from the clinicaltrials.gov website.

For anyone not intimately familiar with the site, one of the chestnuts (at least for me) are the study inclusion & exclusion (I/E) criteria. You may be thinking "Jonathan, interesting, but so what?" Well, if you're tasked with developing External Control Arms (ECAs) a few queries can better inform the Real World Data elements you may need to find.

("For a more in-depth look at ECAs, we recommend this article: 'The Next Horizon of Drug Development: External Control Arms and Innovative Tools to Enrich Clinical Trial Data' by Zou et al.." https://pmc.ncbi.nlm.nih.gov/articles/PMC11043157/)

Traversing Google Gemini had a lot of ups and downs. You will see that it gave me the tools to explore how I want to explore and in the end it gave me the information I wanted, albeit not by doing how I wanted it to get it.

*warning, long blog alert.

Here's the query that I fed Google Gemini and its response:

Well, that was not what I had hoped to get back. But then it surprised me with this:

What you don't know, is that I had been pestering Google Gemini for this information over several days and was trying to find a way for it to get me what I wanted. It offered up results of fine work by Friends of Cancers Research (FOCR) and ASCO (see the very bottom of this post), but I really wanted to get this information from the source (clinicaltrials.gov). While it didn't "build a house" for me, it did give me a hammer:

And then I took a left turn and decided to challenge it in a different way:

And on and on it went. Giving me steps and code:

The steps and corresponding code is lengthy and I will not provide all of it here. It will take me quite awhile to decipher. If I get there, I promise to share it with you. However, I do offer you this alternative:

And then I needed to check whether it was being truthful or feeding me some "half-truths":