A systematic review of randomised controlled trials (RCTs) of veterinary homeopathy has not previously been undertaken. Using Cochrane methods, this review aims to assess risk of bias and to quantify the effect size of homeopathic intervention compared with placebo for each eligible peer-reviewed trial. Judgement in seven assessment domains enabled a trial's risk of bias to be designated as low, unclear or high. A trial was judged to comprise reliable evidence if its risk of bias was low or was unclear in specified domains. A trial was considered to be free of vested interest if it was not funded by a homeopathic pharmacy. The 18 eligible RCTs were disparate in nature, representing four species and 11 different medical conditions. Reliable evidence, free from vested interest, was identified in two trials: homeopathic Coli had a prophylactic effect on porcine diarrhoea (odds ratio 3.89, 95 per cent confidence interval [CI], 1.19 to 12.68, P=0.02); and individualised homeopathic treatment did not have a more beneficial effect on bovine mastitis than placebo intervention (standardised mean difference -0.31, 95 per cent CI, -0.97 to 0.34, P=0.35). Mixed findings from the only two placebo-controlled RCTs that had suitably reliable evidence precluded generalisable conclusions about the efficacy of any particular homeopathic medicine or the impact of individualised homeopathic intervention on any given medical condition in animals.
Statistics from Altmetric.com
HOMEOPATHY is a system of medicine that uses specific preparations of substances whose effects, when administered to healthy subjects, correspond to the manifestations of the disorder (symptoms, clinical signs, pathological states) in the individual patient. Homeopathic prescribing is thus normally based on the individual's ‘totality of symptoms’ (Swayne 2000).
In (rightly) extolling the importance of placebo-controlled clinical trial design in evidence-based veterinary medicine, Overall and Dunham (2009) make a key comment about the application of the scientific method in the particular case of homeopathy: ‘If homeopathy [wishes] to be considered by scientists, [it] must be shown to be valid using methods that science uses to evaluate all treatment modalities’. This systematic review directly addresses this issue by examining the evidence available in randomised placebo-controlled trials of veterinary homeopathy, using established systematic review methods.
The use of homeopathy in veterinary medicine is highly controversial, with strong viewpoints expressed on each side of the argument (Baker and others 2005, Hektoen 2005). On the one side, homeopathy's proponents point to case reports, non-randomised comparative studies and randomised controlled trials (RCTs) that are perceived to be positive; on the other side, critics highlight the improbable efficacy of very highly diluted medicines, with the conclusion that the research evidence in homeopathy cannot plausibly contain positive findings. Nevertheless, many homeopathic medicines are not in this ‘ultra-molecular’ range (Rutten and others 2013), and the plausibility argument is being approached directly in new research on nanoparticles (Bell and Schwartz 2013) and other physicochemical properties of dilutions (see Hill and others 2009). Moreover, neither side of the argument has been in a position to know the true and full nature of the clinical research evidence as, until now, no systematic review of RCTs in veterinary homeopathy has been conducted.
A previous systematic search of the published RCTs in veterinary homeopathy by the current authors identified 38 peer-reviewed papers that were regarded as potentially eligible for detailed systematic review (Mathie and others 2012b). Such in-depth reviews can inform the debate surrounding the effectiveness of veterinary homeopathy in general or the efficacy of its medicines for particular medical conditions.
In taking forward the first phase of this in-depth review programme, the approach continued to reflect the above literature analysis, and three principal attributes of research design or intervention were distinguished: (a) controlled by placebo versus controlled by other than placebo; (b) individualised homeopathy versus non-individualised homeopathy; and (c) treatment versus prophylaxis. This review reports findings from an appraisal of placebo-controlled, peer-reviewed trials of veterinary homeopathy (individualised or non-individualised, treatment or prophylaxis).
For each eligible RCT, the aim was to assess the risk of bias (Higgins and Altman 2011), together with the direction, statistical significance and effect size of homeopathy or placebo on the main outcome measure. For groupings of RCTs per medical condition, we aimed to determine pooled effect size by meta-analysis and, by reflecting study quality critically in our approach, to determine if (and which) homeopathic interventions were more beneficial than placebo, and for which particular medical condition(s). Practical and reporting standards for systematic reviews set by the PRISMA Group (Moher and others 2009) were adhered to throughout.
All randomised and controlled trials that assessed a homeopathic intervention, in any species except people, were eligible for review according to the inclusion/exclusion criteria outlined previously (Mathie and others 2012b).
The following electronic databases were searched, up to and including March 2011 (during peer review, the systematic literature search was updated up to the end of December 2013): AMED, CINAHL, CENTRAL (Cochrane), Embase, Hom-Inform, HomVetCR (Carstens-Stiftung), LILACS, PubMed, Science Citation Index and Scopus. The literature search strategy has been described in detail elsewhere (Mathie and others 2012b).
Identifying papers for full data extraction
Eighteen papers were identified as satisfying the key acceptance criteria: substantive report of clinical treatment or prophylaxis trial in veterinary homeopathic medicine; randomised; controlled by placebo; and published in a peer-reviewed journal (Mathie and others 2012b).
RTM screened and categorised each of the 18 potentially relevant papers to assess their eligibility for full data extraction. JC independently appraised these decisions. Any differences of opinion were resolved by consensus discussion.
Exclusion criteria before data were extracted:
▸ Research using radionically prepared ‘homeopathic’ medicines;
▸ Intervention tested was homeopathy combined with other (complementary or conventional) medicine or therapy.
Fig 1 illustrates the PRISMA flowchart, which focuses solely on previously identified placebo-controlled trials (Mathie and others 2012b). RCTs controlled by a comparator other than placebo are the subject of a separate review. The 18 RCTs included in this systematic review comprised 12 treatment trials and six prophylaxis trials.
Data extraction and management
Because it is recognised that contacting the original authors of trials may lead to overly positive answers (Higgins and Altman 2011), the authors of eligible RCT papers were not approached for clarification on unclear or missing facets of any of their methods or results; however, original authors' references to their previously published study methods were eligible for follow-up and taken into account as appropriate. For each of two assessors working independently, relevant data were extracted and then recorded using a standardised data collection format (Microsoft Excel; Microsoft).
None of the 18 papers reported more than one RCT. For a paper reporting an RCT that involved more than two groups of subjects, the authors focused data extraction on only one pair of groups as follows: treatment in preference to prophylaxis; and placebo control in preference to other-than-placebo control.
For studies that comprised more than one homeopathy group, the total sample size (and associated outcomes) cited reflected the total number of subjects in the homeopathy groups combined (Higgins and others 2011). This was the approach in all cases, that is, where the same homeopathic medicine was used, and with the same timing of administration, but with different potencies; where the same homeopathic medicine and potency were used, but with different timings of administration; and where a different homeopathic medicine was used.
Assessment of risk of bias
In eligible trials, and using the standard criteria defined by the Cochrane Collaboration (Higgins and Altman 2011), extraction of information enabled appraisal of freedom from risk of bias. There were three options when assessing the trials: ‘yes’ (low risk of bias), ‘unclear’ (uncertain risk of bias) or ‘no’ (high risk of bias).
This approach applied to each of seven assessment domains:
▸ I – Method used to generate the random sequence;
▸ II – Method of allocation concealment used to implement the random sequence;
▸ IIIA – Blinding of trial personnel, including animal owner as appropriate;
▸ IIIB – Blinding of outcome assessors;
▸ IV – Whether all randomised patients were completely accounted for in the analysis;
▸ V – Evidence of selective outcome reporting;
▸ VI – Evidence of other bias, such as extreme data imbalance at baseline.
For the domain relating to whether all randomised patients were accounted for in the analysis, unless there was indication to the contrary, a trial was regarded as being at high risk of bias if there was greater than 20 per cent participant attrition rate, and irrespective of whether intention-to-treat analysis had been carried out on the data.
For the domain relating to evidence of selective outcome reporting, judgement was based on reported outcomes and not on a comparison with an original trial protocol (as none exists in the public domain for RCTs of veterinary homeopathy) but rather on a comparison with the details given in the materials and methods section in the original paper.
The two assessors' judgements were mutually scrutinised and compared, with discrepancies between them resolved by consensus discussion, with the aim of producing summary of findings tabulations to characterise all eligible trials.
Using the Cochrane approach, each trial was designated as one of the following: at low risk of bias (free of bias in all seven standard domains of assessment); at uncertain risk of bias (unclear judgement of bias for one or more domains, and no evident risk of bias in any domain); or at high risk of bias (evident risk of bias for one or more domain) (Higgins and Altman 2011).
For each trial that was not found to be at high risk of bias (that is, it did not attain a ‘no’ response for any domain), the evidence was seen as reliable if it was assessed to be free of bias in each of four domains: randomisation; blinding of trial personnel; blinding of outcome assessors; and patients accounted for in analysis. These criteria for ‘reliable evidence’ are analogous to those used by Shang and others (2005) in their designation of ‘trials of higher methodological quality’. According to these criteria, the latter group included adequacy of allocation concealment (domain II) but excluded completeness of data analysis (equivalent to domain IV). In the veterinary trial context, domain IV was regarded in this study as a more relevant arbiter of reliable evidence than domain II.
For each trial (for the purposes of risk-of-bias assessment and for assessment of treatment effect), the main outcome measure was identified using a refinement of the approaches adopted by Linde and others (1997) and by Shang and others (2005). The main outcome measure of each trial was based on a hierarchical ranking order (consistent with the World Health Organization [WHO 2002] classification system for levels of functioning linked to health condition):
– Treatment failure;
– Pathology; symptoms of disease;
▸ Health impairment (loss/abnormality of function, including apparent presence of pain);
▸ Limitation of activity (disability, ill health);
▸ Surrogate outcome (for example, blood test data).
In cases where, according to the judgement of the reviewers, there were two or more outcome measures of equal greatest importance within the above hierarchy, the designated ‘main outcome measure’ was selected randomly from those two or more options by tossing coins or rolling dice.
The single endpoint (measured from the start of the intervention) associated with the designated main outcome measure was taken as the last follow-up at which data were reported for that outcome. The exceptions to this were two ‘semi-crossover’ trials (Hektoen and others 2004, Werner and others 2010), for which the single endpoint was taken at seven days after treatment commenced and before any cross-over. A semi-crossover study design is one in which patients defined as non-responders after a pre-defined period of time are re-randomised to another arm of the trial.
Summary effect measures for main outcome
For each eligible trial, effect size was taken to be the difference between the homeopathy and the placebo groups at the predetermined endpoint of the trial as follows (Mathie and others 2013):
▸ For dichotomous measures: odds ratio (OR) with 95 per cent confidence interval (CI);
▸ For continuous measures: standardised mean difference (SMD), with 95 per cent CI.
For a trial in which the selected dichotomous measure was presented solely as percentage data (tabulated or graphed) per group, the categorical data required for analysis were calculated from the available published information. For a trial in which the selected continuous measure was presented as a mean, but without an associated standard deviation (sd), Cochrane-recognised methods were used to calculate or estimate sd per group (Higgins and Deeks 2011).
If the original paper did not provide adequate information on the designated main outcome measure to enable data extraction, that trial's outcome was classified as ‘not estimable’ and a further potentially estimable outcome was not sought.
Mean effect size was interpreted as follows: an SMD of less than 0.40 was considered ‘small’; an SMD of 0.40 to 0.70 (inclusive) was considered ‘moderate’; and an SMD of more than 0.70 was considered ‘large’ (Schünemann and others 2011). Using the standard formula to convert SMD approximately to OR (Schünemann and others 2011), the corresponding effect size thresholds were calculated as: an OR of less than 2.10 was considered to be ‘small’; an OR of 2.10 to 3.60 (inclusive) was considered to be ‘moderate’; and an OR of over 3.60 was considered to be ‘large’.
Under the separate group headings of individualised homeopathy and non-individualised homeopathy, and for each of any subcategories in which there was more than one RCT paper with extractable data, the authors aimed to determine summary statistics, using meta-analytical methods, for the following:
▸ Disease-specific prophylactic effects per species;
▸ Disease-specific treatment effects per species.
All calculations and analyses were performed using Review Manager 5.2 (Cochrane). Given the anticipated heterogeneous data for intervention effects, the random effects (rather than fixed effects) model was planned for each meta-analysis (Deeks and others 2011). For each meta-analysis, the need to merge dichotomous and continuously variable data, using the Cochrane-recognised method, as required, to re-express SMD in terms of OR, was also anticipated (Schünemann and others 2011).
Reflecting overall study quality
The main focus was on the data extracted from trials that were not designated to be at high risk of bias, and especially those that were deemed to contain reliable evidence.
The focus for primary conclusions was trials with reliable evidence that were also not explicitly funded, directly or indirectly, by a homeopathic pharmacy (that is, there was no overt vested interest in the trial's findings).
Direction of effect of treatment/prophylaxis per trial
For a conclusion that homeopathic intervention impacted on health outcome (that is, statistical significance favouring homeopathy, at P≤0.05), the following were required:
▸ Dichotomous measure (OR): lower 95 per cent confidence limit of 1 or more;
▸ Continuous measure (SMD): relevant 95 per cent confidence limit of less than or equal to 0 or more than or equal to 0, depending on the direction of the hypothesis favouring homeopathy.
Table 1 provides details of each of the 18 eligible trials: (i) individualised homeopathy/treatment (two trials); (ii) non-individualised homeopathy/treatment (10 trials); and (iii) non-individualised homeopathy/prophylaxis (six trials). Data presented include: medical condition, species, nature of the homeopathic intervention, and trial setting. The tabulation also includes details of the RCT's source of funding, together with the associated freedom from vested interest: only one trial (Holmes and others 2005) was deemed to be clearly free of such vested interest.
The trials were clinically extremely diverse. In the 18 eligible studies, four different species are represented: cattle (10 trials); dogs (two trials); goats (one trial); and pigs (five trials). Eleven different medical conditions are represented.
Table 2 details sample sizes, designated main outcome measures, and trial endpoints of the included trials. Table 2 also accounts for three trials for which data were not extractable for analysis. Diversity was again apparent, with large variations in sample sizes, main outcome measures and the timing of the trial endpoint.
Risk of bias
Table 3 details risk-of-bias judgements per trial.
Some of the papers were written to such a poor standard that risk-of-bias assessments were not straightforward. However, consensus discussion always resolved the matter. Only one trial had a low risk of bias; six trials had uncertain risk of bias; and the remaining 11 trials were judged to be at high risk of bias, some of them failing the assessment criteria in more than one domain.
Of the seven trials not deemed to be at high risk of bias, four of them failed to meet the criteria for reliable evidence. These were Holmes and others (2005), Hielm-Björkman and others (2009), Guajardo-Bernal and others (1996) and Soto and others (2010). One of the three remaining (reliable) trials had potential vested interest due to funding source (Cracknell and Mills 2008) and so was not reflected in the primary conclusions.
Considering the assessments overall, high risk of bias was evident across all domains (Table 3). Domains IIIA (personnel blinding) and domain VI (other biases, which most frequently were connected with extreme data imbalances) were at high risk of bias particularly frequently. For randomisation (domain I), only seven of the 18 trials had low risk of bias. A problem that was frequently encountered was the lack of detail or clarity provided in the original papers; allocation concealment (domain II) was associated with the greatest rate of uncertainty in assessment.
Trials at low/uncertain risk of bias
The summary statistics for each of the seven RCTs that were assessed to be at low/uncertain risk of bias are presented in Table 4. The direction of effect was towards homeopathy in each of the six trials from which data could be extracted. The seventh trial in this category (Holmes and others 2005) contained logarithmic data. Due to the diversity of medical conditions, species and types of homeopathic intervention displayed in these six trials, and the fact that analysis was limited to data from trials at low/unclear risk of bias, it was not considered appropriate to carry out meta-analysis on disease-specific treatment or prophylaxis.
Trials designated as having reliable evidence and being free of vested interest
Only two trials were considered to have reliable evidence and contributed to the primary conclusions (that is, low/uncertain risk of bias overall, with low risk of bias for each of Cochrane domains I, IIIA, IIIB and IV, and also without overt vested interest due to funding source); these were Hektoen and others (2004) and Camerlink and others (2010). An additional facet of study design should be noted in each case: the Hektoen RCT was a semi-crossover trial, whose data were extracted for the precrossover timepoint; and the Camerlink RCT involved treating sows but evaluating piglets.
These two trials analysed different medical conditions, species and categories of intervention, and so a meta-analysis was not appropriate.
As displayed in Table 4, the Hektoen trial (A01) showed a non-significant treatment effect in cattle with mastitis (SMD –0.31, 95 per cent CI –0.97 to 0.34, P=0.35), while the Camerlink trial (A24) showed a statistically significant effect in the prophylaxis of diarrhoea in piglets (OR 3.89, 95 per cent CI 1.19 to 12.68, P=0.02).
Trials at high risk of bias
The 11 trials that were judged to be at high risk of bias are presented in Table 5. The direction of effect was towards homeopathy in five trials (three statistically significantly, though highly variable and imprecise in the magnitude of effect size) and towards placebo in four (none statistically significantly). Data were not extractable from the remaining two trials.
Although six trials with extractable data were judged to be at low/unclear risk of bias, their diverse characteristics prevented the application of meta-analytical methods to examine disease-specific effects. Each of these six trials had a direction of treatment effect towards homeopathy, three of them significantly (P≤0.05). However, such ‘vote counting’ masks a lack of robustness in the data, the small sample size per trial contributing to a wide confidence interval with lower limit approaching null effect. A meta-analysis of all six trials together (irrespective of species, medical condition, outcome measure or type of homeopathic intervention), following the combined analytical approach reported by Linde and others (1997) and Shang and others (2005), was outside the scope of the current review. Sensitivity analyses on relevant groups of trials, reflecting the full range of risk of bias across all 18 RCTs, is the subject of a separate paper (Mathie and Clausen, in press).
Only two trials did not have high risk of bias and contained reliable evidence that was free from vested interest; the disparate nature of these trials again prevented meta-analysis. It is the evidence separately from those two trials, therefore, that forms the basis of the primary conclusions of this review: Hektoen and others (2004) (individualised treatment using unspecified homeopathic potencies of acute bovine mastitis); and Camerlink and others (2010) (prophylaxis of porcine diarrhoea with an ultra-dilution of the homeopathic preparation Coli).
Camerlink reported a large, though imprecise, effect size that was statistically significant, in favour of homeopathy; the smaller effect size reported by Hektoen was not statistically significant. The idiosyncrasies of study design in these two trials (indirect treatment of piglets via the sow; semi-crossover RCT) should also be noted. From the only two trials that inform the primary conclusions, therefore, there is modest evidence that homeopathic Coli has a prophylactic effect on porcine diarrhoea and that individualised homeopathy is not more beneficial than placebo in the treatment of bovine mastitis. It is not possible to generalise from these two divergent findings to other medical targets of veterinary homeopathic treatment or prophylaxis, for which there is no reliable (or any) research evidence.
The authors are confident that the risk-of-bias assessment approach was fair and rigorous, always erring on the side of stringency. Indeed, the definition of an RCT with reliable evidence closely approximates that used by Shang and others (2005) to designate a trial of ‘higher methodological quality’; moreover, source of research funding (vested interest) was also taken into account as a matter of key importance in these assessments. Given that the 18 eligible trials were published in peer-reviewed journals – arguably therefore representing the ‘best’ of such RCT research in veterinary homeopathy – the inability of our findings to yield broader-based conclusions may disappoint homeopathy's advocates and critics alike. Neither gloom nor glee is indicated here for either side of the polarised argument: matters will be settled only in the event of more and much better-quality research. Similar cautionary notes about the limited quality and clarity of the available evidence in homeopathy were applied in a recent Cochrane review – involving one of the current authors – of homeopathic Oscillococcinum for influenza in humans (Mathie and others 2012a).
Shortcomings of the papers included in this review were concerned particularly with randomisation, allocation concealment, blinding and source of funding. Such failings were undoubtedly contributory to the extraordinarily variable results provided by the 11 trials that were assessed to be at high risk of bias. A further problem was the unclear original reporting of the key data, particularly for standard deviation of a continuous measure: in such cases, the use of recognised Cochrane methods to calculate or estimate the sd enabled the otherwise unusable findings of some trials to be included in the data presentation (see Tables 2, 4 and 5). Deficiencies in RCTs of conventional veterinary medicine are also well recognised, including the potential relationship between funding source and positive outcome reporting (Wareham and others 2013).
The use of a single, hierarchy-based, outcome measure per trial harmonises with the method adopted in major systematic reviews of homeopathy RCTs in humans (Linde and others 1997, Shang and others 2005), as well as in the authors' own study protocol for meta-analysis of RCTs in human homeopathy (Mathie and others 2013) and in quality assessment of RCTs in conventional medicine (Hartling and others 2009). Some commentators have previously expressed concern that trials in homeopathy often use irrelevant or subjective outcomes (Merrell and Shalts 2002). None of the main outcome measures used in this analysis can be regarded as clinically irrelevant: only two (fear response to fireworks [Cracknell and Mills 2008] and ‘pasty’ stools [Kayne and Rafferty 1994]) can be regarded as subjective. Outcome measures that can be regarded as surrogate only featured in trials whose data proved to be not extractable for analysis (Andersson and others 1997, Holmes and others 2005, Danieli and others 2009). It was noteworthy that 16 of the 18 RCTs assessed here did not designate a ‘main’ or ‘primary’ outcome measure and this concern corresponded with the absence of a prospective power calculation to determine the appropriate sample size of trials.
The deficient nature of the current RCT literature in veterinary homeopathy indicates a clear need for further primary research investigation, whether in the context of individualised or non-individualised veterinary prescribing. New research of this nature can be informed by research-targeted observational studies on the outcomes of homeopathic treatment in cats, dogs and horses (Mathie and others 2010a, b). Naturally, such RCTs should be independently funded and strive for reliable evidence, and they should also optimise the model validity of the homeopathic intervention as state-of-the-art (Mathie and others 2012c). An updated search of peer-reviewed RCTs published since an original literature analysis in 2011 revealed only one new placebo-controlled trial (Notz and Hässig 2013). While this means that this systematic review is barely compromised by the absence of up-to-date findings, it also illustrates the near-static nature of this field of research. The authors are currently progressing a separate systematic review of veterinary RCTs in which the control group was a comparator other than placebo.
Although few in number (as well as deficient in quality), the preponderance of homeopathy RCTs in livestock might reflect the needs of the organic farming community, especially in the European Union (IMPRO 2012). The very small number of homeopathy RCTs in companion animals might reflect the difficulty of recruiting sufficient numbers of subjects for individualised treatment or prophylaxis, which necessitates long consultation times with each animal and its owner. The dearth of RCTs in individualised homeopathy per se is consistent with this conjecture.
From 18 RCTs in total, low or unclear risk of bias was ascribed to seven diverse trials, two of which were judged to contain reliable evidence and were free of vested interest due to funding source. Mixed findings in these two trials preclude generalisable conclusions about efficacy of a particular homeopathic medicine or the impact of individualised homeopathic intervention in any given medical condition in animals. There is an obvious need for new and higher quality research.
Conflict of interest statement
As a research physiologist (RTM) and a biologist (JC), each of us is employed by a homeopathy charity to clarify and extend an evidence base in homeopathy. We have applied the normal high standards of scientific method in the conduct of the work and of complete and transparent reporting in the write-up of the paper.
The authors would like to thank Elizabeth Baitson (British Homeopathic Association) for statistical support and Daniela Hacke (Karl und Veronica Carstens-Stiftung) for library assistance.
Provenance: not commissioned; externally peer reviewed
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.