SPEAKER_00 0:00

You're listening to Confidence to Thrive is a podcast for ambitious healthcare practitioners and entrepreneurs brought to you by LTID. Public Confidence to Thrive, a podcast helping phone practitioners and healthcare entrepreneurs navigate the challenges and risks of regulation while building brands that matter. Every episode, Christopher Cloak Brown tackles a different issue facing healthcare practitioners and entrepreneurs, or interviews a guest who is working on the leading edge of private healthcare, building something that matters in their sector or profession. My name is Jody Rainsford, and on today's episode, I'll be discussing with Christopher about the opportunities and challenges posed by AI for healthcare practitioners and business owners and how to navigate the obvious as well as the hidden risks many may not have considered. Welcome, Christopher.

SPEAKER_02 0:44

Thanks, Jodie.

SPEAKER_00 0:45

Okay, so obviously we know AI is transforming healthcare at an incredible pace. But with a lot of that innovation comes a lot of challenges, especially when it comes to things like liability and risk management. We normally start off with the risks, but I thought we'd kick off this one hearing about some of the opportunities that AI may afford private healthcare practices and some of the things that are happening. And of course, you have your finger on the pulse of everything that's happening in the industry. What do you see as the reason for all the excitement around AI? Like what have you seen and what have you seen that it could do for healthcare businesses focused on growth?

SPEAKER_02 1:24

Yeah, uh, I should probably start off with putting my cards on the table. So I did do I did do a PhD in machine learning a very long time ago. Um it's not so much the technical AI stuff uh that's been done today, but it does mean that I was initially very skeptical of the whole AI movement. Uh my skepticism has reduced dramatically or disappeared to some degree. I think I have a better understanding of what's what it is that's been created now and what it can be used for.

SPEAKER_00 1:56

What was the skepticism about?

SPEAKER_02 1:58

I mean skepticism, which all gets a bit of intellectual and very technical is there's the there's no I in AI. Oh, right. That all gets down into more philosophical questions about what is intelligence and how does intelligence come about, all sorts of stuff. Um but leave those leave all that aside and just look at what the thing is and what it does, basically. And uh the fact that there is no I is important to the whole insurance thing, and we will get back to that. Uh, but if you look at AI as a very sophisticated support tool for a human that can increase human productivity, I would say, in the right applications for at least 10, if not 15, times, uh then it has enormous potential in business. And I imagine you can make your staff 10 times more productive. Uh, it's just massive. And certainly in medicine, uh there's just dozens of uses. Um the most obvious one, which I think increasingly everybody's going to start to use, is just in the straight note taking. Uh, if that's just purely transcribing, or whether that's trying to actually create some more comprehensible notes out of it. So actually being able to take contemporary notes, which are sort of well written and even well structured, and things like that would in many cases be a huge leap forward. That's great. That's one piece. There is uh you can see it in terms of diagnostic support. So the doctor can look at it and feed the symptoms in, and the AI can start to suggest what it thinks it might be, and the doctor can sort of like your classic interface with pick your favorite AI can adjust the prompts to say seems you know left, right, up, down. I think I think it's more of that and less of this, and see the AI can certainly help that thought process. Uh, there's a whole area, so some of our clients who it's particularly cancer treatment, they're now AIs who will suggest off-license use of existing medications to treat cancer. And that comes down to I think that's fairly patient specific. Uh, again, there's sort of tools and AIs which look at the efficacy of medications against certain people and certain genome types. How great is your imagination? I think if you can imagine it, there's an AI use and they're exploding all over the place. So, yeah, it will have a massive impact, that's for sure.

SPEAKER_00 4:23

One area you spoke about there was note-taking. And if there's one theme that goes through all of these episodes is the importance of note-taking and maintaining note-taking to a very high standard. Is there any uh concern around with AI note-taking? That the usual process of keeping good notes and managing good notes is that constant element of review, whereas in a lot of the situations where AI is taking notes that you probably don't need to look at them again until of course the next time or a time after that, which of course is great in terms of saving time and everything else. Is there ever going to be a situation that's going to arise where potentially, especially as well as with a diagnostic aspect as well, that liability could potentially shift from either the doctor prescribing or the doctor saying notes to the provider of the AI because it hasn't performed in the way it's supposed to perform? Are are those things already being discussed as to how exactly that would play out?

SPEAKER_02 5:32

Yeah, so yes. And I did find myself on a panel discussing uh insuring AI the other day. That was mainly because a friend of mine was running the panel, and somebody who knows a lot more than me dropped out at the last minute. So I found myself in just a distinguished company. So one of the debates there is to say, and this is the issue. So people like Munich Ree, some very clever people at Munich Ree, uh, take the view that you can insure AI, but the way you insure AI is it is a statistical engine and it has a error rate. And if you test it enough, you can get a pretty good sense of what that error rate is, and then you have you have the error rate and you can estimate what the financial cost of the error is, and that's all the information you need to ensure it. So that's great if you view AI in that lens. Put it back into that medical context, though. And medicine is still where you have a professional giving a professional judgment, and they cannot delegate the responsibility for that professional judgment. So the first thing people need to understand is the buck, if you're the head doctor, the buck stops with you, whatever the AI did in its notes or said about the diagnosis or whatever, it's your notes and your decision, and that's where the buck stops, and you can't delegate that responsibility, so you can't often say, Oh, well, the AI just didn't quite get it right in that sense. The AI writing the notes wrong is no different from you getting your junior doctor to write the notes and them getting it wrong, it's indistinguishable. The other factor, and this goes back to the sort of philosophical question as to is AI, is it intelligent? The point is the liability is a professional liability, it is for a failing in your professional duty or professional care, and that professional duty and professional care comes about because of your knowledge and intelligence and judgment gathered over time. So, and if you think about how medical negligence is tried, it's tried in this idea of a Bolum test. So you are put on the witness stand, you have expert witnesses called, and the court opines on whether or not you acted to the standard of a the appropriate standard for a medical professional in those circumstances. So until you can put your AI on the stand and get it to justify its actions and what it did, and then have the court determine whether or not it's negligence. I don't think that you can actually you either have to change the way that negligence works, uh, or you can't have AI as a doctor, it's not making those decisions. And again, when you get into all that deep intellectual stuff, AI doesn't work like the human mind, basically. There's sort of fundamental differences, and I don't think, and one of the things uh about AI is it doesn't have this sort of self-knowledge, uh it does it doesn't have this internal representation of its own knowledge that it can then manipulate and justify and talk about and so on. So you can't put it on the stand and and test whether it's acted up to the appropriate professional standard. Uh so you just can't do it. So at the moment you can't ensure it because you you can't it just can't stand trial, you can't do it. So it's a little bit stuck in my point of view.

SPEAKER_00 9:13

And so where can people use it? Where can it be used until that question is resolved?

SPEAKER_02 9:20

Oh, it's a support tool. So long as you understand I am the treating physician, and I am going to use this array of tools at my disposal to support my practice, what I do, uh my decision making, and so on, but it is always my decision making. Uh you still need to have the physical doctor there. So you do now have the AI apps, I'm sure, and it's like I don't know, some take a photography of your face and it will tell you what's wrong with you or whatever it is. So those things again are uninsurable and interesting. I mean, it's a whole regulation. I mean, medicine's moving so fast now. You then get into the question of, well, are you diagnosing? And if you're diagnosing, then do you need to be a qualified doctor? Which the machine might have been fed with every medical degree course in the world or whatever, but it's still not a qualified doctor.

SPEAKER_01 10:14

Yeah.

SPEAKER_02 10:15

So uh can it do that? What are the regulations uh around that? And again, it then comes back to well, you know, even if it passed all the medical exams, can I make it a medical doctor because it can't answer for its actions if it's sued? Even in driverless cars, that's my objection to driverless cars. The cars driving all around, tactic's driving around San Francisco, driverless cars drive themselves around, and it's fine, they work. But the thing you can't do is the moment one of those plows into a family on the pavement, you can't put it in the dock and ask it how it decided to plow into the family rather than the llama or whatever it was. You can't test its moral judgment and make a decision on its choices. Again, you're gonna have a problem the day that happens.

SPEAKER_00 11:07

One of the things about the the rush to AI compared with, say, other uh changes in uh computation and software and things like that, is that it's the wideness of the accessibility that literally anyone can now go and use this in a way that allows them to uh shortcut things to transcribe, do any of these things uh within businesses, within schools, within practices, in in in ways that there's no gatekeeping to it. There's no, oh, there's a professional standard here that we need to adhere to. You can't use this, you can use this. Someone in a practice may be using it uh rather innocuously for something, and there may be other potential issues. What other potential issues uh are thrown up that people don't think about? Obviously, there's the the judgment there element, but uh what about things like data protection and thinking about where the information that's been input into these models is being used and where it's going?

SPEAKER_02 12:06

Yeah, so data protection is the other sort of massive issue that people don't generally consider. Uh, and again, you need this sort of massive compute that isn't available on your phone or whatever you're accessing these uh systems through. Uh so it all goes through to the servers in the sky and who knows where they are and who controls them and what data's stored and uh what data's accessed and even which jurisdiction it's in. So, again, even if you're comfortable with the general idea of the the provider and the servers, if the servers go outside the EU, then you need you either need to go to a country with EU data rule compatibility, or you need extra checks and balances and measures. And when you just fire up an app on your phone, you should be honest, you've got no idea where it's going. That whole data protection issue is a massive consideration at the end of the day. If you are recording a consultation that is a piece of the patient's medical records, it's a medical history, it's a medical notes, uh, that in GDPR world is special information, and that's disappearing off into the ether to be used and abused wherever, unless you know otherwise.

SPEAKER_00 13:17

And so is the answer to that to perform uh some kind of assessment of potential risks whenever you're thinking about using a particular technology?

SPEAKER_02 13:30

Yeah, oh absolutely. Uh so the first thing you need to do is to have a fairly detailed view of what data has been captured and where and how that is being stored and under what rules. We had a client who consulted us on a software that purported to be able to detect stress in somebody's voice. So you talked to it, spoke to it, and and it would tell you your how stressed it thought you were. Now they contacted us because I think it was an Australian company, but their servers were in Taiwan. There's a choice between Taiwan and Singapore for the servers. So Singapore would have been alright. Singapore meets the uh GDPR data standards. Uh so we were sort of happy with there wasn't an awful lot of personal information, and most of that was only stored fairly short term, and then it's all in a country with the appropriate data standards. Taiwan is a completely different kettle of fish in that it doesn't adhere to the EU data standards, so you then need to put all sorts of uh contractual items into you know you have to ensure that your GDPR rights are installed in your contract, and then you can even start to worry about whether that contract means anything in A, Taiwan Taiwan, or B, if you take the view that at some point Taiwan will become part of China in China. Uh it's a whole kettle of worms, it's something that medics generally are not trained to think about, haven't needed to think about. Uh it's another thing we support our clients on through that in detail. And I can't remember, I think the outcome was that they were happy to do it, but on the basis that the servers were in Singapore, I think was the outcome of that.

SPEAKER_00 15:10

What are the other kinds of issues that people are bringing up around AI? Because it sounds like a lot of these aren't necessarily ones that they are bringing up, they're ones that emerge because you've had a conversation and the the mention of AI or the mention of data has been put across your desk, and so with a bit of further investigation, emerging issues that no one would have thought about before.

SPEAKER_02 15:35

I think that's the thing, is people don't think about people that cause up an excitement, and sure it is exciting, and there's sort of amazing things going on, and they don't think about a lot of the flip side of it. So I think the first AI question we ever had was one of our sort of weight loss surgery businesses, and uh unfortunately they don't use them anymore because it sounds awful and they don't really work. One of the ways they originally started to think about dealing with the weight loss without surgery was basically he got a balloon and he swallowed the balloon and then he inflated this flipping balloon inside you. The company that provided those balloons had created an AI app to support their patients after they'd swallowed these flipping things, and our clients had been asked to trial this AI app. Now, their concern was that one of the side effects of swallowing these things is you can actually find it difficult to swallow water for a while afterwards.

SPEAKER_00 16:36

Quite essential.

SPEAKER_02 16:37

Yeah, well, yeah, so but not immediately a problem. So the normal advice is keep trying, and eventually the there must be an inflation device. I've never seen one, I haven't gone near the things, but it must be it must be quite a chunk to get down. So keep trying, and eventually it all sort of subsides and you can swallow water and it's all fine. But yeah, as you've just suggested, after 36 hours and you can't swallow water, you start to become fairly hydrated, and you need medical attention quite quickly. And so their question was well, if we go from a patient one of these on a Friday afternoon and off they go for the weekend, and the answer phones in emergency and so on, but we're not generally available, and this app just says, Oh, keep trying, and I come Monday morning, they're going to be horribly dehydrated. Is that our fault? So the first answer is yep, that's your fault because you can't delegate your duty of care. And two, do you really trust the app to actually understand that subtle difference? Because if you just train it on data, the advice is just keep trying. The app doesn't understand, and this is that intelligence piece, it doesn't have that sort of model of a person to say, as you immediately said, because you've got the model of the person. Well, if you don't drink for too long, it's a real problem. And that's a sort of classic example of where the issues come up. Uh being a couple of yes, I'm sure I told you the other story about uh a client and now a friend, he's a really lovely guy. Uh he's a very successful uh cosmetic surgeon, plastic surgeon, 40-year career. Uh, he's got a new business where he's he's gone for the AI thing, everything's got AI in it, the note-taking's got AI in it, the sort of what different implants will make you look like is all powered by AI. There's AI front to back. And I was invited to an absolutely lovely sort of celebration of his 40 years career with all the people he knew and this lovely setting and this lovely party, and this going on about AI, and maybe I need to improve my social etiquette because I said, Oh, have you told your insurer about all this AI?

SPEAKER_01 18:43

Because I don't think it'll be covered. Anyway, I've after security escorted me out.

SPEAKER_02 18:49

I know it was it was fine about it, but it's yeah, it's it was uh it really brought home how people could get absolutely caught up in in all the AI uses, but hadn't thought about what else insurance people think about all the time? What's a risk? What's how do I mitigate it? How do I deal with it?

SPEAKER_00 19:06

But presumably, for let's say generic insurance providers who don't have a specialism in in any of these sectors and don't have an awareness of it, there's the it's going to be a straight no for a lot of these things simply because the risk is too high. And that considering the amount of opportunity that AI affords, surely that is going to that's gonna hold back a lot of businesses who there are you like you say, there are so many applications for this. It just needs to be viewed with an open conversation with an insurer and for them to have an understanding of where those challenges are.

SPEAKER_02 19:50

Yeah, so I think the thing is it's not holding the businesses back, they're just ploughing on. And at some point there's going to be a very awkward AI incident that is going to be uninsured, and everybody's gonna be looking awkwardly at each other. So I think the way of dealing with that is you still need to absolutely think of the AI input as supporting a person and keeping all that judgment with a person. That's very much in this sort of professional services area. As I say, there are Munich Ray's that out there and is happy to uh is already ensuring AI, but it's insuring AI on the basis that it's a statistical engine, it's a statistical tool. So if you're using your AI, if it's running a plant and sometimes it will flip the thing down the wrong line, and that has a consequence and that has a cost, then fine. People like Munich Ray and the great extraordinarily smart guy who's running it, insurers can get their head around that and they can ensure that. And that's happening today. It's only when you go into this decision making where the eye becomes the critical part. Is it intelligent? Is it making the decisions in this professional services context that it becomes problematic?

SPEAKER_00 21:08

In an area such as aesthetics, the use of AI isn't just necessarily about showing the changes that could be made to your body, your face, whatever. It also becomes quite a powerful selling tool as well. Because you previously, however, you described the impact it would have, you would have to use your imagination to some extent. You might use examples, there would be a kind of a gap, and there would it be very clear that the result may not necessarily be exactly like that because all results are different. Whereas AI is very different in the sense that it can put something in front of you that looks very convincing and it is almost selling it as well. Is there a challenge around that where uh AI goes from being a tool to help explain something or explain changes to something which is actually quite persuasive in terms of its uh how it's being used.

SPEAKER_02 22:03

I mean, I I guess that's the question, isn't it? And that that's going to be that's something that the practitioners are going to have to be careful and wary of. So absolutely. So Adrian, the plastic surgeon, was telling us on in his business they they have an AI tool that will give you a model of what it thinks you will look like with these implants versus those implants and so on and so forth. So enormously sophisticated and much more visual and so much clearer as to what the expectations of the results will be. If that's more accurate, that's maybe a good thing because in cosmetic surgery one of the biggest risks is unrealistic expectations. So if you have a very realistic if you can make the view of the outcome more realistic that could be good. People don't have such a high expectation of thinking oh if I have this done it's not it's gonna it's gonna in their imagination yes yeah actually whereas when you're actually presented with this is what it's going to look like it could counteract that. But so long as that view is accurate if you get an overoptimistic AI that misrepresents what the outcome might be then that's a bad thing because then you are giving people unrealistic expectations and that can lead to claims.

SPEAKER_00 23:17

The fact is that it's so new that those things aren't going to play out for a while whether that's happening again a lot of that even with that being able to handle complaints and claims in the right way is going to be the key to heading those particular challenges off.

SPEAKER_02 23:35

And some of that has to come down to the surgeon's experience so the surgeon can do all this modelling but if he sees something that looks completely unrealistic he's got to either play with the prompts to get it more realistic or not show it or whatever. So you can't just say oh this is what you're gonna look like because he still needs to put that filter on it so that the it's his responsibility that the patient does not have unrealistic expectations of the outcome summing this up if you were a business and you found yourself in a situation where you have started to implement AI but you possibly haven't informed your insurance provider what steps should you take now to make sure that you are fully protected the thing to do is to just list out all the AI you're using, what you're using it for, where you're using it and have a chat with the people who arrange your insurance to start with and say is this okay and get an honest opinion and feedback and sort of really think through how it works. In in my mind there's nothing in AI that can't be made to be insurable and to work so long as you think about it in the right way. And that really comes down to this whole idea of the tools are not the decision maker the tools support the decision maker so you just need to ensure that the to how those tools and the decision maker are interacting to come to a result at the end of the day in that professional services setting it is still the professional the decision maker who decides what to do who is going to be sued not the AI I think that's a good place to to to leave it there.

SPEAKER_00 25:23

And I'm sure with something like AI we're gonna have a situation that in the months ahead new situations are going to arise new challenges going to arise things that people haven't thought of or you're gonna probably get a phone call from someone you're gonna have a new case where you're just like wow I've not seen I've not seen this before just because the speed at which everything is moving is so fast. I'm sure we'll revise this in a future episode so it just remains for me to say thank you very much Christopher and look forward to speaking to you on the next episode. Thanks Jodie thanks for having me. Thank you for listening to Confidence to thrive before you go please rate review and subscribe to Confidence to thrive on your preferred podcast platform and help us spread our message to others who are making a difference in private healthcare this podcast was brought to you by Ouritity insurance advisors who support your business ambitions. Our litity advises practitioners, owners and entrepreneurs of healthcare practices on mitigating risks so your business can thrive. Learn more about howitzity can support you by finding the link in the show notes or visiting ourlitity.co.uk