5 use cases for network meta-analyses you should know about

Transcript

[00:00:00] Alexander: You are listening to the Effective Statistician Podcast, a weekly podcast with Alexander Schacht and Benjamin Piske, designed to help you reach your potential lead great science and serve patients without becoming overwhelmed by work. Today I’m talking with Daniel Saure about five use cases for network meta-analysis you should absolutely know about.

So stay tune, and now some music.

Now, over the years, we have created a lot of free content for you, all kind of different things about data visualization, about influencing about. Whatsoever. And now we have collated all this content in one library for you to access. So head over to the Effective Statistician homepage, look for the library.

You’ll find it in our new homepage, and then sign up there and get access to all the content and also future content will all post there. So go over to the effective decision and sign. Set. I’m producing this podcast in association with psi, the community dedicated to leading and promoting the use of statistics within the healthcare industry for benefit of the patients.

Join PSI today to further develop your statistical capabilities with access to the ever-growing video demand content library for ere registration to all PSI webinar and much more. Head over to psiweb.org to learn more about PSI activities and become a PSI member today.

Welcome to another episode of the Effective Statistician. Today I’m talking to a good friend. Daniel, how are you doing?

[00:01:56] Daniel: Hi, Alexander. Thanks. I’m doing good. Very much looking forward to today’s episode.

[00:02:01] Alexander: Yes. Today we are talking about a really interesting topic. Something that is and don’t wanna say haunting me , but following me for a very long time.

And these are indirect treatment comparisons as and especially also network meta-analysis. How we can all use these. I think there’s very often only a, very limited point of view on all the opportunities here. But before we dive into the kind of heavy methodological application part of the, of this episode, Daniel, maybe you can introduce yourself a little.

Yes. Thanks. Yeah, I’m Daniel. I’m located in Germany, so I’m a yeah, mathematician by training and yeah, have been working for, I think, more than 10 years now in in data science, medical biology. Yeah. I think Alexander, we work together for around four years for Eli Lilly in Yeah. Late phase statistics, medical phase affairs, H d A and so last September, I, yeah, joined Beringer Ingelheim, so as a data scientist and I’m still active in the yeah, h d A phase.

And yeah, very much looking forward to today’s episode, I think. Today, it’s already the second time . I’m happy to be here. I think I, I had a look yesterday the first episode was about everything you need to know about matching, adjust indirect comparison. I think we recorded it almost two years ago yeah.

Yeah. And you also participated in the. 200th episode.

[00:03:35] Daniel: Ah,

[00:03:35] Alexander: that’s true. Yep. So that means your episode was also quite successful. Yeah. In terms of download numbers. So that’s really good. And I’m pretty sure today will be another great one. So let’s talk about. Where we can all use network meta-analysis and for sure for things.

But let’s, for the moment we’ll focus on the medical affairs and h d a kind of applications. What’s the first one where kind of, you know, in the life cycle where we will need to use network

[00:04:12] Daniel: meta? Yeah, I think the first one from a timing perspective also is that you compare your phase two, phase three data with the competition.

I think this is related to the clinical development phase. I think in this phase, your company is developing a new compound. . So maybe in a new indication, maybe a line extension or maybe even the first indication. And so how is this done usually? So you are running one or more than one? Yeah.

RCTs, which are usually placebo controlled or standard of care controlled. And disinformation then is submitted to regulatory author. and beyond that you might be interested in how your compound compares was the active competition. Yeah. I think your data or your trials tell you in terms of efficacy and safety, how do you compare was the standard of care.

But if you want to go beyond. I think, yeah, indirect comparison and network meat analysis are yeah, co common methods which then come into play and yeah, this usually depends heavily on which competitor traits are already there, so which information you already have. So in some cases you, you might be able to compare where is.

One other active treatment, depending on the market. In other cases, maybe you’re entering an already big market, so then you already have a big amount of active treatments available, and then in that case you might be able to even perform a network meter analysis. Yeah.

[00:05:44] Alexander: Yeah. Completely agree. I think it’s really important, both internally and externally.

 To frame your data. Yeah. Yes. And to show how successful is it actually. Yeah. Of course, assuming that you have a positive phase three study or positive phase two study, but how positive is it? Yeah. Is it, far zen compare than what’s. ? Or is it similar? In which extent is it better?

Yeah. , does it work faster? Does it, is it, creating higher levels of efficacy? Or is it even, is it safer here? It’s really about differentiation. And of course if you are first to market, you are never really first to market. There’s usually some kind of treatment out there and could be standard of care and then could be different things here and potentially even you could bring here in also some observational data.

If there is really no kind of established new compounds. Yeah so if all compounds that are currently used to maybe not be studied for particular lesions, indication and it’s just used for setin clinical practice, I think it’s also important. To go out there and proactively answer these questions because otherwise we get into the second use case and that is where you are.

Maybe not the, you see a competitor actually, publishing something and then you are end the situation like, oh, there’s this new truck and holds it doing against art. And maybe, on first glance, it looks really, and then I’ve seen more than once organization panicking. Yeah. They can say, oh, company X, Y, Z has just published a results and they look so great.

As you look into the kind of, nuances of some methods and say yes, if we would have done the same, we would also look pretty good. Yeah. . But you can only do that if you then have, indirect comparisons and all these kind of different things available. Yeah,

[00:07:48] Daniel: That’s true. I completely agree.

And I think what occurs in this case, and also to the previous case, is that everyone is comparing the results anyway. Yeah. Yeah. So for example if you go to a medical conference, So maybe they’re a poster on treatment, a showing response rate of 70%, and then another poster or a different compound showing a response rate from 60% and everyone is saying, oh, treatment A is 10% or has a 10% higher response rate.

But this might be the case in reality. Yeah, and I think methods like in indirect comparison and yeah, NMA really help here to give better estimates on this in case you don’t have head to head data. And yeah. So in coming back to this case, when you really compare your or the data to competitive data with your own product, I think you can really make use of all possibilities where new data can arrive.

So this can be posters or presentations on conferences. It can be medical publications. It can be shown in HDA doses, which. Yeah. Open for the public, and I think here really is, it’s again, important to Yeah, proactively identify the case and to prepare and then you can really, yeah. Make a difference when you apply such methods.

Yeah. Yeah.

[00:09:05] Alexander: Imagine there is new, presentation at a conference. Yeah. And you have everything ready on your pc and you basically, just as you see the presentation type ins the numbers from the slides. Yeah. Yeah. And directly after the presentation, you can, show to your colleagues.

You have, we have just seen these kind of things and this is how it looks in comparison to what we have. That can have an amazing impact. And that way you can show. And demonstrate your value to the organization. , and this will help you to get a seat at the table. That will help you to become more influential, become known for someone that is has this competitive SAVs and insights and can, do these kind of things that lots of people actually don’t Underst.

[00:09:57] Daniel: Yeah, that’s true. And yeah, I think this is just another possibility where you can really help to inform. Not only internal, but also external decision making. Yeah. On your compound strategy. And yeah, coming back to your point to get a seat on the table, I think this is really then where wherever functions are struggling, we are to do the next step based on new data.

And I think you can really work together here proactively with medical marketing, sales and other functions. . Yeah. To make a difference. Yeah.

[00:10:28] Alexander: One of the things is important. It’s not, you don’t only need the data. , you don’t only need the results. What you also need to provide is training around the method and explanations that around the method.

What are the strengths, what are the limitations? , what can you say based on your, these network analysis, what can’t you say? Yeah. . So it’s always an important to, to be prepared and say, okay. Here are limitations in terms of, these different design characteristics and that has, can bias in this direction or in set direction, that makes it difficult to make this comparison?

Or is that comparison, these kind of training slides stuff you can have basically pre-prepared, yeah. So that, when’s these question. Come and say, will come, said you have, already answered that. Oh, potentially you have maybe even discussed that before the data comes, said, you know the, already the design of the other competition study and because of that you can say, okay, they will likely have.

Better, worse, different results since what we have done.

[00:11:34] Daniel: Yeah, I agree. And I think the key here is really to plan for those activities. Yeah. Cross-functionally not only, in your department or function. Think obviously you need to plan for the resources. Yeah. Because this is something which is not, I think not a standard activity across companies properly.

This and we are all busy anyway, but I. When you can really see the value of this, I think it’s worth to, to do it and to plan accordingly for this. Yeah. And as you say I think it’s not about to to push the button to run the analysis. I think it’s really to come together cross-functionally, even beforehand to.

To identify the competitors that are of interest. And to see which data you can use then, or which data you expect actually. Yeah. And then after you have to result, it’s then really to draw conclusions of it. Yeah. Yeah.

[00:12:23] Alexander: Completely agree. Yeah. This is third case. We are you can get, creates. Internal panic thing. And that is actually a case where it’s much more difficult to be prepared for, what is this case?

[00:12:39] Daniel: Yeah. So this here, it’s a case whenever you have published any kind of data. So through publications, through presentations. At conferences, yeah. Others can make use of it.

Yeah. In a accord, in a bad way. So there. Third parties I don’t know, medical guidelines or even the competition, who then yeah, will have the possibility to incorporate your data into their own indirect comparisons or network media analysis. Yeah. And then it’s really the question, what do you do if those findings are misleading or harming your product? because I think one, one key aspect of indirect comparison or NMA is that you have consistency across results of indirect evidence and direct evidence. So from head, head tried, and if this is maybe not given what do you do then?

[00:13:27] Alexander: Yeah. I have experienced that firsthand during a launch of a product where a third party.

Couple of third parties were publishing network meta-analysis on a nearly quarterly basis. And whenever these come up Yeah. The companies said that Brexit came out best, was then using that in their for their communications. Yeah. And here it’s an of course important to be able to react fast.

Yeah. Yeah. That you can very fast say okay. They. Included C studies or excluded C studies, they have, lumped up these kind of doses whatsoever. And we have a kind of task force ready that can look into these kind of things, can provide ranks and limitations, can provide recommendations in terms of how to use that and all these different things.

Yeah, so basically you need to have something like communication cascade. And communication plan ready for these situations because they just create a lot of movement overall in the market and then, and a lot of noise, especially when they come from kind of high profile organizations like , if you are the European Association of X, Y, Z, or the us. The American Association of X , and they publish something like this that usually have quite a lot, has quite a lot of impact. .

[00:14:52] Daniel: Yeah. And then you need to answer then to those external Yeah. Publications and how do you do this? Yeah. I think you, you can do it on the one hand in a qualitative way.

 Yeah. To but maybe even better. If you can come up with your own analysis Yeah. Which maybe focus on other aspects and maybe even expands the differences. Yeah. Yeah. So therefore I think there are some possibilities. You you have to act then and again, indirect comparisons and network meet analysis, I think play a key role. Then again, in this case.

[00:15:23] Alexander: Yeah. You can show for example, by rerunning the analysis and ch changing. Some aspect of it, you can show how robust or how not robust they are. Okay. The first case is probably the most used one, and when we had a discussion among peers and a little bit of a survey across industry set, both by far the number one area and we’re pretty much, yeah. Everybody needs to have it more or less by design.

[00:15:54] Daniel: Exactly. And that’s about HDA submissions and payer nutritions. Yeah, which just think an external requirement when you look to HDA bodies worldwide. Think most of them accept and ask for. Indirect evidence in order to calculate the price based on yeah.

Economic models and yeah. This is a given. Yeah. After your regulatory approval, you need to submit to local h d a bodies or to pay us in order to get market access and the price for your product. Yeah. And here again, indirect comparison, network meta analysis plays a crucial role for this. Yeah.

[00:16:33] Alexander: Yeah. And usually it’s not just a report that you once produce and send, it’s done. Yeah. Yeah. But very often you will have some kind of stagger approach in terms of launching products. Yeah. Maybe you , you go first with the UK or was Canada, and then, you have maybe after you have France and then Poland and lots of other countries around the. And very often they will say you need to have an updated systematic treasurer review. time. And just for that reason, you need to update it. The other reason could be is that they say for in our country we have these. Guidelines and therefore you could, we only want to see this selection of treatments.

Or only these doses or whatsoever. Yeah. , I actually live worked on this. I was joking with my colleague on, the scientist that was responsible for diseases. See the n is never dying. It’s kinda . It’s. It lives and needs to be re-updated again and again. I dunno how many versions we had at the end, but quite a lot.

[00:17:38] Daniel: Yeah. Co completely agree. Yeah. I think this is because when you look to the countries, they all have several or different requirements. So there is not one approach that fits all. And I think this won’t change in future because, there is this EU HTA process ongoing and starting in around two years.

Where you will need to submit EU dok or an HDA DOK on EU level, so which will properly comprise several pcos. So this autoimmune then that you need to be ready to conduct several NMAs. So with different. With different population, with different comparators in it maybe based on different outcomes.

Yeah, several times. And therefore I think the situation we have now won’t really change. And yeah, again, I think it’s, yeah, just important to, to have a good setup. To do NMAs

[00:18:29] Alexander: Yeah. PICO stands for population, intervention, comparator and outcomes. Yeah. Little bit of evidences vocabulary.

Yeah. The other point is especially in areas where there is a lot of money involved, there is an incentive for HTA bodies to reevaluate the. Because prices usually go down, and so after there’s a couple of additional therapies entering the market, it can be easily occurring. There’s an, a revenue of all these different trucks.

And there’s a reassessment. And so that’s the fifth place where you need to work on network meta-analysis.

[00:19:15] Daniel: Yeah, that’s true. I think once you’re on the market you really need to constantly update your evidence and reevaluate it and provide it to h d a bodies to pay us. Yeah.

Because the landscape is changing. Yeah. So on the one hand as you have just said there, new mar new treatments will enter the. So therefore the data needs maybe will change. Maybe you create your new own data, so maybe long-term data or you have a additional phase for trial running.

So therefore yeah, think you do not only have difficulty to provide an n m a to local meets, but also changing needs over time. Yep.

So it really makes sense. To invest in training on their animes. Yeah. Which really makes sense to, if you outsource see networks metaanalysis to work, with one partners that can help you do these kind of things in a consistent way, in a fast way.

Again and again because as we just walked through these five cases, where you need to work on network current analysis and these just five cases. That doesn’t mean that you do the NMA five times . Yeah. Could do it for 20 times or 30 times. So there is again and again, these kind of things happen and it starts already, while you are in clinical development because of course you can also use it to inform.

[00:20:40] Alexander: The study designer. Yes, I know many companies do that. Okay, very good. So the five cases were, first understand your phase two, phase three data. So second. Understand competitive data. Third critique, published network meta-analysis and the force and fifths, HDA submissions and reevaluations by HTA. Daniel a great episode. Again, what are your key takeaways?

[00:21:11] Daniel: Yeah, thanks. I think the key takeaway really is that, NMAs in the comparisons are an important tool, not only in development phase, but maybe especially in late phase. So including HTA and medical affairs work. , I think it’s important to plan for those kind of analysis and this not only in your function, but in a cross-functional manner.

So I think all the relevant functions need to be on board. What might be the value of this analysis? What might be the consequences? I think you really need to show how this analysis can help to drive business decisions. Internally, but also externally. And yeah. I think the last point is that you need to have in place good routine to train on this, because in this phase, as we were talking about, Not only you who can run those analysis, but there are several others, your competition, third parties who can run the same kind of analysis, and this is very likely to get tricky over time.

When you have 10 published NMAs with maybe the same set of treatments and different results, then I think you need a lot, you need to put in a lot of effort to really understand. Those difference and to help others what to do with it.

[00:22:30] Alexander: Yep. Completely agree. Having a good training communication pieces is absolutely critical. And finally, it also gives you an all, some opportunity to increase your influence, honestly, reoccurring topics. Thanks so much for the great discussion. Really enjoyed it and we’ll surely speak again on this podcast.

[00:22:53] Daniel: Yeah. Thank you for having me again.

[00:22:56] Alexander: I really hope you enjoyed this podcast episode with Daniel. I certainly did. Head over to the effectivestatistician.com to find the library of all the free resources. Don’t forget about this. So this show was created in the association with PSI. Thanks to Reine and Casey who helped with the show in the background.

And thank you for listening. Reach your potential, lead great science, and serve patients. Just be an effective statistician.