Homeopaths are delighted that a draft report has been released - not sure why

tl;dr - Homeopaths are cock-a-hoop that a report has been released containing the phrase "encouraging evidence" (for homeopathy) but several mentions are just saying that "the term 'encouraging evidence' has not been defined" and highlighting problems with the conclusion. There are 6 'positive' mentions (in favour of homeopathy) in the report. All have been corrected with an annotation highlighting that it was a mistake.

---

Here's the draft first report from 2012, it's 294 pages long (293 pages plus a one page statement at the front from the NHMRC's CEO Anne Kelso saying that -

"NHMRC strongly encourages interested members of the community to refer to the 2015 NHMRC Information Paper: Evidence of the effectiveness of homeopathy for treating health conditions. Contrary to some claims, the review did not conclude that homeopathy was ineffective. Rather, it stated that “based on the assessment of the evidence of effectiveness of homeopathy, NHMRC concludes that there are no health conditions for which there is reliable evidence that homeopathy is effective.”"

The 2015 report (all 2015 docs can be found here) has an accompanying statement with some fairly strong warnings about NOT using homeopathy.

Table of Contents
1. How frequently does the phrase "encouraging evidence" appear?
2. How does the phrase "encouraging evidence" appear?
3. Full details of how "encouraging evidence" is used on each page - 13 to account for
4. Conclusions and rationale for this post
5. Background to the report
6. References
7. Screenshots of search strategy and results

1. How frequently does the phrase "encouraging evidence" appear?
Searching in the 2012 draft report (using Ctrl+F or Command+F on a Mac) for "encouraging evidence" brings up 12 mentions of 13 instances (one isn't picked up in the automatic search). Four are on p9, one each on pages 10, 11, 12, 54 and 236 with two instances on pages 85* and 138. By the way there are also 25 mentions of "no convincing evidence".

*Of the two mentions on p85, one is not picked up by the search. One is a mention, the other an annotation cancelling it.

2. How does the phrase "encouraging evidence" appear?
There is a rough split between statements in the report suggesting that there is encouraging evidence for homeopathy and statements that are annotations, highlighting that the conclusion was incorrect. It's not a 1:1 mapping of a positive phrase with a negative annotation because other phrases are also used (eg "See comments on issues with this section of the report on page XX").

Seven mentions are positive (all are corrected with annotations), 2 are neutral and five are negative. 

Three instances of the negatives say
"The term 'encouraging evidence' is not defined in this report. There are inconsistencies as to how it is interpereted." - examples found on pages 9, 85 and 138.

3. Full details of how "encouraging evidence" is used on each page - 13 to account for
All page numbers are those counted in the PDF by Adobe eg p10 of 294 refers to page "ix" in the document and p85 of 294 is actually page 72.

3a. Page 9 - four mentions
Page nine says that there is encouraging evidence for the effectiveness of homeopathy (one mention) for fibromyalgia, otitis media, post-operative ileus (first time to flatus), upper respiratory tract infection (URTI) in adults and side effects of cancer treatment. All are given 'Grade C'. The annotations (two mentions) point out the flaws - that 'encouraging evidence' isn't defined and that Grade C has sometimes been used to mean 'encouraging evidence' and sometimes 'no convincing evidence' so it has been applied inconsistently.

The last mention is the phrase "listed above under encouraging evidence [Grade C]" which I've taken to be a neutral statement (one mention).

3b. Pages 10-12 - three mentions
Context: Table 1 in the report is a 'Summary evidence of effect of homeopathy by condition' for 27 conditions. The phrases "no convincing evidence", "inconclusive / equivocal evidence", "insufficient evidence to reach a conclusion" or "no evidence" appear next to all but two.

Page 10
Cancer treatment symptoms (one of three mentions) says that there is encouraging evidence for topical calendula and Traumeel S. The annotation says "See comments on issues with this section of the report on page 40". [NB page 40 = p53 of 294]

Page 11
Fibromyalgia (two of three mentions), annotated with reference to page 70. [NB page 70 = p83 of 294]

Page 12
Upper respiratory tract infections (URTI) (three of three mentions) - annotated with reference to page 139. [NB page 139 = p152 of 294]

3c. Page 54 - one mention
There is one positive mention of "encouraging evidence for topical calendula... and Traumeel S" but this is then trashed in the comments. The annotations are a bit brutal including "incorrect - see below" and "The conclusions for topical calendula and Traumeel S cannot be justified AND WERE NOT CONFIRMED BY THE EXPERT COMMITTEE" (their capslock). The annotations also point out that the Traumeel S study was based on a small clinical trial of 32 participants. I don't think they're very impressed.

3d. Page 85 - two mentions
One mention of encouraging evidence for fibromyalgia treatment with a Grade C given. The other mention is the annotation saying that the term isn't defined and is inconsistently interpreted. Another annotation says that "this conclusion is flawed" and goes on to say that it should be a D.

3e. Page 138 - two mentions
One mention suggests encouraging evidence for postoperative ileus and this is immediately thwacked with "not defined / inconsistencies".

3f. Page 236 - one mention
This part of the report contains all abstracts of all the references considered / used, with a comment below.

"Although there is some encouraging evidence for hypnosis, herbal medicine and acupuncture..." - homeopathy is not actually included in this bit at all. It is subsumed into the phrase "there is insufficient evidence to suggest that other CAMs are effective for the treatment of childhood conditions". Pleasingly this is one of Edzard Ernst's papers (Ref 43).

4. Conclusions and rationale for this post
So there we have it. The draft first report, released to such fanfare from homeopaths, does not support homeopathy. The NHMRC does not recommend its use in any condition. Since homeopaths will probably try and use this report to suggest that regulatory bodies should loosen their restrictions on homeopathy (eg NHS and RCVS) I thought it best to have something to point people to saying why this is pointless.

5. Background to the report
A few years ago an Austrlalian study was commissioned by the NHMRC to examine the evidence relating to the use of homeopathy to treat a variety of health conditions. There were a number of problems with the draft of the 2012 report - it included studies of lower quality and was not clear about how it graded the quality of studies.

A second report was commissioned, and later published in 2015 and concluded that the evidence for homeopathy was not impressive. Hardly a surprise given that pretty much all overviews of the evidence come to the same conclusion.

Homeopaths were annoyed that the earlier 2012 draft report wasn't published. They were under the impression that the first report had shown some success for homeopathy and that this had been deviously edited out in the second report, unfairly implying a negative effect for homeopathy.

The first draft report has now been released and homeopaths are very happy that the phrase "encouraging evidence" appears in the document, such as -

"...in which the author concluded that there is “encouraging evidence for the effectiveness of homeopathy” in five medical conditions." ⁽¹⁾

It would be more correct to add the word 'wrongly' or 'mistakenly' between 'author' and 'concluded'.

Appended at the start of the draft report is a statement from the CEO of the NHMRC, the text of which is at the top of this post. The HRI has managed to draw this conclusion from it -

"We also welcome the valuable clarification provided by NHMRC CEO Prof Anne Kelso, that NHMRC’s second Homeopathy Review published in 2015 “did not conclude that homeopathy was ineffective”, despite claims to that effect in media reports and by anti-homeopathy campaigners." ⁽¹⁾

Incidentally there is an important but subtle difference between these two statements
1. There is evidence that homeopathy does not work
2. There is no evidence that homeopathy works

6. References and further reading
(1) NHMRC finally release first report on homeopathy (27 August 2019) Homeopathy Research Institute https://www.hri-research.org/2019/08/hri-statement-on-release-of-first-report-by-nhmrc/

See also Hinner Feldwisch's thread https://twitter.com/hfeldwisch/status/1166440559386517510

7. Screenshots of search strategy and results
Click to enlarge any that need it.

Ctrl+F or Command F brings up the Find option

Click the gearwheel to bring up the full search options

This is what I searched for in the document - encouraging evidence

There were actually 13 instances but only 12 appeared in the automated search. All are accounted for in my post.

The annotations for page 9 were particularly brutal.