Shor-Posner 2004: Statistical significance and alpha

Shor-Posner report that they performed statistical evaluations, "with alpha level at 0.05". What does that mean?

Short version: If there is less than 5% probability (1 chance in 20) that the result they got from their study was due to chance, rather than to a real treatment effect, they will consider their result to be "statistically significant". The way we get that 5% number is by converting the 0.05 that Shor-Posner gives us (5 out of a hundred) to percent notation, or 5%.

Longer version: Because we can't observe the mechanism of cause-and-effect directly (except in a few very particular cases), we have to make inferences, or reason, about the connection between the results we see and the treatment we are testing.

But inference is tricky--just because something happens many times in a row does not guarantee that it will happen again next time. If I see 100 patients for whom massage is effective in relieving pain, I may be confident, based on experience, that it will work for the 101st patient as well--but I cannot guarantee it, because I can't directly observe the mechanism. So if I want to create a study to test whether massage reduces pain, I can't say I will show that it reduces pain in any number n of people, and then guarantee that it will work in the n + 1th person--induction just is not robust enough to guarantee that.

What I can do is take advantage of an asymmetry (unevenness) in the way logic works. While I can't prove from a repeated positive result that something will work for everyone, I can take advantage of the fact that all it takes to disprove something logically is one example to the contrary. In other words, here is the asymmetry:

Hypothesis 1: Massage reduces pain

Patient 1: true, Patient 2: true, Patient 3: true, ... Patient n: can't guarantee in advance before I test on this patient

Hypothesis 2: Massage causes no change in pain (the null hypothesis)

Patient 1: false

Already with my first patient, I have disproved my null hypothesis, and thus strengthened (not proved) the opposite of the null hypothesis--in other words, I have shown that it is false that massage causes no change in pain, and therefore, I have strengthened the opposite hypothesis, that massage causes a change in (reduces) pain. (In reality, I would do this for many more than just one patient, but it's the same idea.)

You see the trick? Because the two situations aren't symmetrical, I can take advantage of that fact, couch my hypothesis as the null hypothesis, and try to disprove it, rather than pursuing the impossible goal of proving a positive hypothesis.

And that's where alpha comes in: alpha is how much error I am willing to accept in rejecting the null hypothesis. I can never totally eliminate the possibility that what I am seeing is pure chance, but I can make it very, very small--my alpha level of 0.05 (5%) or less. I could make it 0.02 (2%) or less, or 0.001 (0.1%) or less--whatever is appropriate. The point is that once I have set that level, it is my threshold for accepting or rejecting the null hypothesis. For Shor-Posner, if the probability of her finding occuring by chance is less than 5%, she will reject her null hypothesis (massage has no effect on the immune system of HIV+ Dominican children), and will consider her results--that massage does have a positive effect on their immune systems--statistically significant.

Shor-Posner 2004, Methods

Shor-Posner and her team applied for and got the standard approval from the Institutional Review Board (standard for studies that involve human beings as subjects of the study), and began the study in June 2003 in the Dominican Republic.

The inclusion criteria (characteristics the children must have in order to be included in the study) were between 20 and 8 years old, confirmed HIV+ infection, being treated a an infectious disease clinic (Robert Reid Cabral Children's Hospital) in Santo Domingo, and their parent or caregiver gave signed consent. Exclusion criteria are not mentioned.

She mentions that the children were randomized to either a massage treatment group or a friendly visit control group, although she does not describe how the children were randomized.

Although the massage is referred to as a "structured protocol", they do not give details, just calling it "moderate pressure stroking and kneading of muscles, using a non-scented oil"--so, basically, effleurage and petrissage. This article is a very short review, so presumably that is why they did not go into detail on the protocol; they probably will in a longer article. Since one of the cornerstones of the scientific method is repeatability, you really should be able to repeat a study, based on the investigator's description of the protocol. Clearly, this is not enough information to do so, and so our ability to evaluate and replicate the protocol is limited.

The control group was, as often is the case for massage research, a 20-minute visit with the nurse, where they read, talked, or played games 20 minutes at a time, twice a week for 12 weeks.

The team drew blood at the start of the study to determine a baseline level of lymphocytes (including CD4/T4 and CD8/T8 cells), and again after the 12 weeks of the study to determine if any change had occurred.

They report that they performed statistical evaluations, "including comparisons between means and proportions using one-sided Student's t-test and Fisher's exact tests, with alpha level at 0.05". For our purposes, we are not going to worry about the details of what these tests mean (although if you want to get involved with carrying out massage research, you will need to know more about statistics to design a study). All that we will concern ourselves with here is the alpha level, and its relation to statistical significance. That's a topic important enough to be the subject of its own post, coming up tomorrow.

Shor-Posner 2004: Introduction, resumed

In the Introduction, the authors describe the problem they want to study ("what?"), and the importance of the problem, or their motivation ("why?"). To show their connection to what other researchers have studied, and how their research fits into that big picture, they also do a brief review of the scientific literature.

Shor-Posner's "why":

In developing countries, where antiretroviral medications are not yet readily available to slow disease progression, massage therapy may have the potential to provide an important, safe, and sustainable form of immuno-stimulation.

Context: AIDS (Acquired Immune Deficiency Syndrome), caused by the human immunodeficiency virus (HIV), eventually destroys the immune system of the infected patient, leaving him or her vulnerable to infections that a healthy person could fight off.

AIDS is not an equal-opportunity killer--people in the developed world have access to many resources that people in the developing world do not. Early diagnosis, access to drugs, good nutrition--all of these, while they cannot cure AIDS, can dramatically improve quality of life and length of life expectancy for people who have access to them. By contrast, if people do not have access to them, these factors can magnify each other--getting diagnosed late means less time on drugs (if they are available at all), and poor nutrition gives the immune system another obstacle to fight. So it is not particularly surprising that infection and death rates for AIDS infections are higher out of proportion in the developing world, compared to the developed world.

Although accurate data is difficult to come by, The the Joint United Nations Programme on HIV/AIDS estimates that anywhere from 270,000 to 760,000 people in the Caribbean are living with HIV/AIDS. They estimate that in the last year, around 52,000 people got infected with the disease, and 35,000 people died from it.

According to Claire-Cecile Pierre, MD of the Cambridge Health Alliance, in 2003, somewhere from 350,000 to 590,000 adults and children were estimated to be living with HIV/AIDS in the Caribbean, while from 790,000 to 1.2 million were estimated to be living with HIV/AIDS in North America. They estimate that in that same year, from 30,000 to 50,000 adults and children with HIV/AIDS died in the Caribbean, and from 12,000 to 18,000 died in North America.

Although those numbers are approximate, we can make a few scenarios from them. Let's first assume the worst-case scenario: the most infections and the most deaths. For the Caribbean, that means 50,000 dead out of 590,000 infected (8.5%), and for North America, that means 18,000 dead out of 1.2 million infected (1.5%).

Now let's figure the best-case scenario: the fewest infections and the fewest deaths: For the Caribbean, that is 30,000 dead out of 350,000 infected (%), while for North America, that is 12,000 dead out of 790,000 infected (%).

Best-case scenario: North America--1.5, Caribbean--8.6%

Worst-case scenario: North America--1.5%, Caribbean--8.5%

Despite the differing possible minimum and maximum numbers, the rates are fairly consistent. In other words, adults and children living with HIV/AIDS in the Caribbean die at 5.67 times the rate (8.5%/1.5%) that their North American counterparts do.

It is in this context of higher rates of disease and death, and less access to resources such as standard drug therapy that Shor-Posner and her team decided to investigate the effects of massage on the immune system of Dominican children infected with HIV-1.

Shor-Posner's "what" (her team's research question):

The present study investigated the effectiveness of massage therapy in enhancing immune status in HIV-1 infected children living in the Dominican Republic, who are particularly vulnerable to rapid disease progression due to the limited access to antiretroviral therapies, prevalence of malnutrition, and increased risk of opportunistic infection.

Shor-Posner's literature review ("what other researchers have done to date"):

Daily massage therapy has been shown to enhance natural killer cytotoxic capacity in HIV+ adults (Ironson et al., 1996) and improve markers of disease progression (CD4 cell count, CD4/CD8 ratio) in adolescents who received massage treatment twice weekly for 12 weeks (Diego et al., 2001). Less frequent (once weekly) massage, however, has not been associated with immune improvement in HIV infected persons (Birk et al., 2000).

I'm going to reinterpret this in the form of a concept graph, a way of visually summarizing and representing complex information. In our concept graphs, the circles will represent "things", such as "effleurage" and "pain" [nouns]:



Adding lines between the circles will represent relationships, such as "decreases" or "increases" [verbs]. The line starts at the thing that causes the relationship, and points at the thing which is affected by the relationship:



So far, we have two things (effleurage and pain), and some kind of relationship between them, but we don't know what that relationship is. We need some kind of relationship description to clarify this question, so we add a "down" arrown to mean "decreases". The following concept graph means "effleurage decreases pain".



We're almost there--the last thing we need is an indicator of the meaning of that relationship. After all, the same thing can mean different things in different contexts: lowering the blood pressure of someone with high blood pressure is a desireable result, for example, while it's not so desireable for someone whose blood pressure is already low. So we add an indicator (a "smiley-face") to indicate that this "decreases" relationship between effleurage and pain is a good outcome:



So applying those ideas to a concept graph of Shor-Posner's literature review, we get the following overview of the effect of massage on immune system markers:

Ironson 1996:





Some positive results, some neutral--the zeroes show that the massage had no effect, and the neutral faces mean that that fact is neither positive nor negative.

Birk 2000, neutral results:



and Diego 2001, positive results:



So in this post, we have learned:

• the researchers' motivation: the HIV+ children's lack of access to antiretroviral drugs
  
• their research question: to study the effectiveness of massage in promoting the children's immune systems
  
• the positive indications and the neutral indications in the research literature--why they think that massage may be an effective intervention, based on the work that has gone before
  

It's been pretty slow going, and if you have stuck with it this far, I appreciate it! But remember, as these concepts get more familiar to you, you'll be able to read these posts and the associated articles faster and faster. They won't all be so slow as our first couple of ones.

Here's the Dominican Republic, where the study took place, in relationship to Miami, where the research group is located:





(Source: http://worldatlas.com/webimage/countrys/namerica/caribb/do.htm)

Shor-Posner 2004, resumed

(Since we're going to finish the Shor-Posner article, I have moved it up from the original posting below, in order to keep all the parts of the discussion in sequence. In discussions with Dorothy, my editor on the book, I've come to the conclusion that my original way was too scattered, and that an entire "thread", or discussion of a topic, should occur together to make it easier for the readers to follow particular discussions. So I'll move the earlier discussions of Shor-Posner 1998 up here, to recap, and then we'll finish the article before starting any other discussions.)

Our first MJC article is Shor-Posner 1998: Massage treatment in HIV-1 infected Dominican children: a preliminary report on the efficacy of massage therapy to preserve the immune system in children without antiretroviral medication. It appeared in the Journal of Alternative and Complementary Medicine in 2004 Dec;10(6):1093-5.

The authors are

• Gail Shor-Posner, PhD
  
• Maria-Jose Miguez MD, PhD
  
• Maria Hernandez-Reif, PhD
  
• Eddy Perez-Then, MD, MPH
  
• MaryAnn Fletcher, PhD
  

Drs. Shor-Posner, Miguez, and Perez-Then are affiliated with the Division of Disease Prevention, Department of Psychiatry and Behavioral Sciences, University of Miami School of Medicine, Miami, FL, and Dr. Perez-Then is also affiliated with CENISMI/Robert Reid Cabral Children's Hospital, Santo Domingo, Dominican Republic.

Dr. Hernandez-Reif is affiliated with the Touch Research Institutes (Tiffany Field's research group), Department of Pediatrics, University of Miami School of Medicine, and Dr. Fletcher is with the Department of Medicine, University of Miami School of Medicine.

In addition to the IMRaD structure we discussed below, research articles also typically have an Abstract at the very beginning (or sometimes at the very end). This abstract is an executive summary, which hits the main points of the article very briefly, so that you can decide whether you want to spend the time to read the article in full.

Here is Shor-Posner 2004's abstract:

OBJECTIVES: More than 1.4 million children are living with HIV and global access to antiretrovirals is not yet readily available. Massage therapy, which has been shown to improve immune function in HIV+ adults and adolescents, may provide an important complementary treatment to boost immune status in young children living with HIV disease, especially those without access to antiretroviral medications. No studies have been conducted, however, that specifically target massage therapy to enhance immune function in HIV+ children.

DESIGN: Clinical trial with eligible, consented HIV+ children randomized to receive either massage therapy or a friendly visit (controls).

SETTINGS/LOCATION: CENISMI/Robert Reid Cabral Hospital, Santo Domingo, Dominican Republic.

SUBJECTS: HIV+ children ages 2-8 years.

INTERVENTION: Massage therapy sessions (20 minutes, twice weekly, for 12 weeks), conducted by trained nurses, following a structured protocol of moderate pressure stroking and kneading of muscles, using a non-scented oil. The friendly visit control group, (reading, talking, playing quiet games), met with the nurse twice weekly for 12 weeks.

OUTCOME MEASURES: At the initial evaluation, and following the 12-week intervention, blood was drawn to determine absolute helper (CD4/T4) and suppressor (CD8/T8) counts.

RESULTS: Children in the control arm had a greater relative risk of CD4 count decline (>20%) than massage-treated children (RR = 5.7, p = 0.03). Lymphocyte loss was also more extensive in the controls (p < 0.02), and more of the control group than the massage group lost >50 CD8 lymphocytes (p = 0.03).

CONCLUSIONS: The efficacy of massage therapy in maintaining immunocompetence may offer a viable alternative to the thousands of children worldwide without antiretroviral access.

Don't worry if there are unfamiliar terms and statistics in the abstract--for now, just read it and decide whether you think this article is interesting enough to spend time reading it in detail. We will cover the important concepts as we come across them, and by the time we have examined this article together, they will be familiar to you.

Shor-Posner followup

Tomorrow let's resume where we left off with the Shor-Posner article. In the meantime, I'd just like to add that I received a very gracious email from Dr. Shor-Posner, where she mentioned that they have written an article on the outcomes of their study on massage for HIV+ Dominican children, and they are conducting a study on the massage-induced mechanisms in immune function. I will look forward very much to seeing their results, and will pass along anything I learn about it.

A good editor is priceless

Well, the book is coming along nicely, although more slowly than I would like it to. The cover is probably going to look like the illustration above; I don't expect it to change much at this point.

All of the chapters are almost finished, and several of them are totally finished. The chapters are as follows (some minor rewording may occur, but it is basically set now):

1 - The Research Literature and You

2 - A Word About Words

3 - Science, Evidence, Medicine, and Massage

4 - How We Know What We Know About Massage

5 - Accessing the Medical Literature on Massage

6 - Reading Massage Research--The Structure of Journal Articles

7 - Reading Massage Research--The Introduction Section

8 - Reading Massage Research--The Methods Section

9 - Reading Massage Research--Just Enough Statistics

10 - Reading Massage Research--The Results Section

11 - Reading Massage Research--The Discussion Section

12 - Reading Massage Research--The Abstract

13 - Using the Information in this Book

14 - Going Farther--Getting Involved in Massage Research

plus Glossary, Bibliography, Author Index (who does massage research?), Affiliation Index (where is massage research done?), Journal Index (where is massage research published?), and Topic Index (subjects in massage research). If you will permit me a little pride in my work, I am quite pleased with how the book is proceeding.

One thing that I did not estimate the time for correctly was the editing. I sort of envisioned the editor, Dorothy, scanning what I wrote for typos, and then signing off on it. On the contrary, she is engaging profoundly with the text, reading it with fresher eyes than mine, finding places where I need to explain more clearly, and other places where I need to stay more focused. In other words, she's acting as an advocate for the readers, and making sure that I always meet my obligation as an author to make the book accessible to them. Although the book is taking longer than I originally estimated with this painstaking process, it is nearing the end, and it is worlds better for the process and her input than it would have been with mine alone.

Too long

It's been too long since I've blogged here, and my New Year's resolution for 2006 is to be more regular and dependable (more reliable, in technical terms) in blogging about research in massage.

It's no excuse, but I've had my head down working hard on the reading massage research book. In that process, I come up for air, only to find that I've let over 2 months pass here. So my resolution is to pick up where I left off, and to not let that much time pass again--book or no book.

Happy New Year!