Dr Mervyn Patterson offers advice on understanding terms used in clinical studies
Doctors and nurses providing anti-ageing treatments for their patients should consider the evidence supporting the claims made by the various companies. Practising in private medicine removes many of the checks and balances found within the National Health Service, which protect patients from experimentation and exploitation. Medical professionals should be able to balance their need to make a living, with an ethical requirement to do what is best and scientifically proven for their patients.
An area where there is a great deal of rightful skepticism is the skincare industry, which, in my opinion, tends to excel in marketing jargon and often lacks substantial supporting evidence. A review in 2006 showed that only nine ingredients in skincare products had completed clinical trials, while another 15 ingredients had undergone animal or laboratory testing(1). Another review in 2008 claimed that only eight ingredients had adequate clinical support for wrinkles and 17 herbal products had been shown to produce a visible benefit in treating skin conditions and diseases(2). Of the 8,000 known antioxidants, only 28 had undergone double blind controlled clinical trials and only 14 of these demonstrated efficacy(3,4).
The proof for evidence-based medicine consists of human clinical trials. The gold standard for clinical testing is double blinded, prospective, randomised, controlled, human clinical trials against placebo or an approved prescription product, conducted by a third party research group, using the final finished formulation. In an ideal world, all cosmeceutical companies would provide this sort of evidence, but in reality we see a rather different approach. Phrases such as clinical evidence, clinical studies, dermatologically tested, proven ingredients, pharmaceutical grade, well-researched and cutting edge science are just a few phrases of the hundreds used by companies and their marketing teams to convey the impression that their products are evidence-based and thus effective. Pharmaceutical companies are required to undergo several phases of rigorous clinical trials and the first includes trials involving 20 to 80 people, with the number recruited sufficient to determine statistical significance. The goal of these trials is to get some measure of effectiveness and safety.
Open label vs. single blind vs. double blind: In open label, also known as a pilot study, both the patient and the researcher know which treatment is received. In a single blind study the patient doesn’t know what they are getting and in a double blind trial neither the patient nor the researcher knows which treatment is received. This information can be retrieved in an emergency.
Prospective vs. retrospective: Trials that gather data as they progress are prospective; retrospective trials gather data after the trial is complete.
Randomisation: In randomised trials, a computer decides which patients receive the active product or the control.
Placebo control: One group of patients will receive a dummy product, which looks like the study drug but doesn’t contain the active ingredients.
In vitro vs. in vivo: In vitro are tests done in a laboratory using test tubes or petri dishes. In vivo trials for skincare products involve using human or animal skin at predetermined test sites.
Single centre vs. multi centre: At a single centre trial there is only one researcher group running this study at one centre. In a multi centre study there is more than one research site. These multiple test centres are developed to test for population, geographical and climatic variations.
Split face vs. full face parallel: Split face studies are where half of the face receives the tested product and the other half receives the control product. Full face parallel studies are different in that a proportion of the study panelists receive the test material and the others receive the placebo or active control. A split face study requires fewer patients to reach statistical significance. Full face parallel studies test both groups at the same time.
p-value: This is the measure of how much evidence is needed to compare to no change or no effect. This value is a measure of probability. As an example a p-value of 0.5 is the measure of probability that if the clinical study were to be conducted 100 times, 95% of the time the result would be the same. A p-value of 0.10 means the same result would occur 90% of the time and a p-value of 0.20 means the same result occurs 80% of the time. A p-value of 0.05 or less indicates statistical significance.
Statistically significant: When a result is statistically significant it means that there is enough difference between the two groups to produce a predictable result. Smaller p-values indicate more significant results. A p-value of 0.05 or less is recognised as the cut off point where statistical significance and therefore efficacy can be claimed. This has been determined by multiple regulatory agencies including the EU. The phrase ‘trend towards significance’ is used where the difference between the two groups produced a less than 0.10 p-value.
Statistically vs. numerically superior: In some studies, companies claim superiority over another product or placebo but no p-value is provided. When a claim is made of superiority, look for the p-value; if it isn’t present then be highly suspicious that the results were numerically better but are not significantly better, thus no more effective.
Some companies claim to have clinical data but refuse to provide detail regarding the studies. Often when you look deeper at websites and company information, the clinical evidence turns out to be a series of before and after images. Failure to provide robust supporting clinical evidence should raise alarm bells amongst medical practitioners. Many companies refer to studies on individual ingredients but you cannot assume that activity will be maintained when that ingredient is mixed with multiple other ingredients. Has the biological activity of the ingredient been affected and has incorporating it into a new product altered the concentration? Given the complex formulations and multiple ingredients added to create cosmetic elegance, it is highly likely that many active ingredients could be rendered useless within current skincare brands. Only clinical trials such as the ones outlined above will prove effectiveness. Much of the testing referred to is in vitro but this does not ensure that activity is maintained on real skin or that the ingredient is in a high enough concentration to achieve an effect.
It is important to consider who conducted the study. On many occasions companies will claim sponsored studies that are actually con- ducted by in-house laboratories. This may reduce the chance of the data being truly objective.
The length of the study is also an important issue. For clinical relevance to skin and all skin cells, including melanocytes, the study must be a minimum of 12 weeks and preferably four to six months. Many studies have been published that have been as short as three to six weeks. This time duration is too short to properly assess the skin’s response and safety of the product with chronic use.
Take care with studies where the improvements are measured from baseline but fail to include any comparison to placebo or a pre- scription active. Many studies test products on areas of the body such as the forearm or back and ask you to believe that similar effects will be seen on the face. Given that the skin is significantly different in lipid concentration in different areas, this means that similar results may not be seen. Many companies will refer to tests on the eye area but this skin is very sensitive to changes in expression, humidity and placebo products such as glycerin. In short it is easy to get ‘improvements’ in the eye area but these may not relate to ‘ac- tive’ ingredients in the test product. A study claiming that panelists could see a difference is not significant if experts cannot determine any statistical change. An important issue to consider is who reviewed the data. A summary by a dermatologist or expert investigator will mean that the data has been analysed appropriately. Acceptance of the study in a peer-reviewed journal or on a poster exhibit at international meetings helps to support the validity of the work.
All clinicians involved in using skincare to help their patients achieve significant results must consider not just how effective, but how safe the products are. Regardless of how effective a product is, if it triggers significant disease or side effects then it may not be suitable for use. The minimum in scientifically valid safety testing is a repeat insult patch test on 50 patients to determine if the finished product will cause allergic or contact irritation reactions. This test involves applying the final finished product daily to the skin of the back or inner arm, then covered with semi-occlusive dressing. It is performed on normal subjects with normal skin. The test site is then checked daily for evidence of pink or red colour changes, scaling, blistering, pustules or bumps. Additionally, the patient is asked about itching or burning. If a reaction occurs within the first two weeks it is a contact allergic reaction. If a reaction occurs after this then it is a contact allergen(5). All practising clinicians should question the skincare companies that they intend to use about the safety data surrounding their products.
The most important things to look for in a clinical study
Financial disclosures: Medical director at Eden Aesthetics, distributor of Epionce / Agera skincare, Colorescience mineral make up, and Dermagenesis microdermabrasion and Dermafrac micro-needling machines. @drmervpatterson
Upgrade to become a Full Member to read all of this article.