The estrogen story
has proven once again that data trumps speculation.

Cardiologist Nanette Wenger never believed all the estrogen hype and says there are no numbers to back up the use of many hormone substitutes.

We must question conventional wisdom
even when it’s not comfortable or popular.

People continue to publish observational studies, but they are unacceptable science.

EPI Primer

Epidemiology - Scientific discipline that seeks to identify the distribution and determinants of health, disease, illness, death, and injuries in a population.

Incidence - Number of new cases of a disease in a group over a period of time.

Prevalence - Number of existing cases of a disease in a group at a single point in time or over a period of time.

Confounders - Factors outside an experiment or trial that distort the outcome of a study.

Multivariate and stratification analysis - Mathematical methods to adjust study results to account for confounding variables, such as the age or socioeconomic status of participants.

Randomized controlled trial - Planned experiment to assess a treatment's effectiveness by comparing outcomes in patients receiving the test treatment with outcomes among patients receiving a control treatment (placebo).

Double blinding - A method to reduce outside bias in a randomized controlled trial. Involves randomly assigning study participants to the treatment group or the control group, with neither participants nor researchers aware of assignments.

Observational trials - Studies measuring patterns of exposure and diseasein populations to draw inferences about causes of disease. Especially useful when a controlled experiment would be impractical, expensive, or unethical such as testing drugs in pregnant women or children).

Oncologist Otis Brawley says it's important to consider harm versus benefits of prostate cancer testing.

Epidemiologist John Young discovered that a census error made the cancer disparity between blacks and whites in Atlanta look much larger than it actually is.

In this issue

From the director / Letters
Burden of proof
Big Idea:
  Regenerative Medicine
Moving forward
On Point:
  Smallpox, big risks?

Story by Valerie Gregg
  Illustration by Octavio Perez



In the practice of medicine, the truth can be hard to find. Should menopausal women take estrogen? Is angioplasty or medication better for mild heart attacks? Does a beer a day keep coronaries away?

The pendulum swings back and forth, and the truth is revealed in bits and pieces. Patients clamor for answers. Studies are conducted on cells, animals, and people. Papers are published, and investigators stake their claims in the popular press. Physicians weigh in with opinion and anecdote. Yet the truth remains elusive, fluid, ever-changing.

Good doctors, like good scientists, must be skeptics, says Emory cardiologist Nanette Wenger. A co-principal investigator for a landmark study that led to a stunning reversal of fortune for hormone replacement therapy (HRT) this past summer, Wenger never believed all the estrogen hype.

"The data were never there for most of the claims presented," she says. "The early observational studies that showed cardiac benefit were flawed and interpreted incorrectly."

During the past half century, doctors encouraged many women to start taking HRT drugs at menopause and to continue taking them for years, even for life.

But on July 17, 2002, women in a massive nationwide study called the Women's Health Initiative (WHI) received letters from the WHI Data Safety and Monitoring Board urging them to stop taking HRT immediately. The HRT portion of the study had been halted when early results showed small increases in breast cancer, heart attack, stroke, and blood clots among the women taking HRT. The risks clearly outweighed the benefits, which include fewer hip fractures and less colon cancer.

The news was a bombshell, considering that millions of American women were taking HRT regularly. National newspapers, magazines, and health news web sites declared it the medical story of the year. Debate continues over whether to prescribe the only therapy known to ease the hot flashes and night sweats of menopause. But the National Institute of Environmental Health Sciences recently declared all estrogens to be human carcinogens. This past January, the US Food and Drug Administration (FDA) slapped an ominous warning label on all products containing estrogen. And national bodies like the US Preventive Services Task Force, the American College of Obstetricians and Gynecologists, and the American College of Physicians have issued guidelines urging physicians to prescribe HRT only for severe menopausal symptoms, and then for as short a time as possible.

Wenger says it's her job to question conventional wisdom when unsupported by evidence and to push the envelope, even when it's not comfortable or popular. She is one of many physicians and faculty members to do so at Emory's Woodruff Health Sciences Center, where the scientific process and discovery are alive and well and ultimately lead to longer, healthier lives for patients.

Data dilemmas

Just 50 years ago, doctors routinely made clinical decisions on the basis of observational studies, anecdote, and personal experience with patients. Although absolute truth remains hard to come by, research has become essential to the practice of modern medicine. When data reach a critical mass or standard, judgments can be made about how best to treat patients, prevent disease, and invest finite health care resources.

So what went wrong with HRT?

The problem was in the data, says Wenger, in the way it was gathered, interpreted, and then used to sell a product.

The HRT story gained momentum in 1966, when an enthusiastic doctor wrote a best-selling book called Feminine Forever. His work was subsidized by the manufacturer of the estrogen replacement drug Premarin. The author likened menopausal women to "castrates" and suggested that they take estrogen until the end of life to avoid becoming dull and unattractive to their husbands.

That publication marked the beginning of an extremely successful marketing campaign. By the year 2000, about 6 million American women were taking Prempro, a combination of estrogen and progesterone. Many patients and doctors were convinced the drug could prevent heart disease, memory loss, wrinkles, and a slew of other age-related ailments on the basis of scores of short-term studies in both animals and people.

But those "observational" studies lack the scientific rigor of randomized controlled trials. Observational studies are much less expensive and time-consuming, but they give investigators much less control and often measure indirect evidence of disease, such as cholesterol levels, rather than disease itself. If not interpreted correctly, study results can also be skewed by outside factors, particularly the characteristics of women who were prescribed estrogen.

The FDA never licensed HRT for anything other than the prevention of osteoporosis and management of menopausal symptoms. But as observational data accumulated, suggesting HRT to be protective against heart disease, Wyeth-Ayerst asked the FDA in 1990 to label estrogen as protective against heart disease, the number 1 killer of women. The FDA declined.

But by the end of the decade, says Wenger, "physicians were promising many things about HRT, essentially using off-label indications."

It finally took a randomized controlled trial - the gold standard of medical research - to put HRT into perspective. Wenger was the cardiologist for the Emory arm of the Women's Health Initiative, which included long-term follow-up of 16,000 women nationwide. The subjects were randomly divided into two groups - one taking placebos and the other HRT - to eliminate bias. (Statistically, random assignments to a control group equally distribute bias from outside factors.) Qualities like this make the results of randomized trials more reliable than observational studies because they allow researchers control over external factors like race, age, income, and education that can affect outcomes.

The impact of randomized controlled trials on the practice of medicine is a relatively recent phenomenon. "They have taught us that some of what we had always thought in medicine was simply collective ignorance," says Wenger. "We are now living in an age of evidence-based medicine. And the estrogen story has proven once again that data trumps speculation. And speculation is all we really had with HRT."

Some physicians have complained that government agencies are giving estrogen - a product that gives many severely symptomatic menopausal women a new lease on life - a bum rap. But Wenger says protective measures like warning labels on estrogen products are no different from safety measures for other drugs. "This is reasonable," she says. "It protects women by defining the risks, and it says to drug companies: 'If you think you have a better preparation than Prempro, then test it.' Will they do it? I don't know. For them, it becomes a business decision."

Last fall, the Journal of the American Medical Association published a paper that was crucial to understanding where the early research behind HRT went wrong. The researcher statistically adjusted the results of observational studies for study participants' socioeconomic status and coronary risk factors and found no cardiac benefit from HRT.

"So now we know that the observational studies were flawed not only in theory but also in interpretation," says Wenger.

Lessons learned

Emory cardiologist William Weintraub, director of the Emory Center for Outcomes Research, sees a tremendous lesson here.

"People continue to publish observational studies as if there were no problem with bias, and that's absolutely ridiculous," he says. "It's unacceptable science. It's one thing to do observational studies, but you have to acknowledge their limitations. Observational studies should generate hypotheses rather than final answers."

Weintraub has been involved in many major clinical trials, including those seeking to find out whether angioplasty or drugs are better to treat mild heart attacks and whether angioplasty or cardiac bypass surgery is better for severe heart attacks. He has published numerous editorials in major scientific journals emphasizing that not all medical research is created equal.

In fact, some research questions can never be tested, even in a randomized controlled trial. It would be unethical to have a controlled experiment of many drugs or treatments on pregnant women or young children. Therefore, questions remain about whether it's safe for pregnant women to take antidepressants or very young children to take stimulants for attention deficit hyperactivity disorder. Debate over such questions rages on.

Whether moderate alcohol consumption prevents heart disease is another issue considered to be untested by a controlled experiment. Two years ago, Weintraub, a reviewer for the journal Circulation, was concerned when researchers in Boston attempted to publish the conclusion that moderate alcohol consumption reduces cardiovascular events among diabetics, based on results of the Physician's Health Study, a large observational study of US physicians with long-term follow-up.

"When they originally wrote the article, they wrote it as if it were definitive proof. As a reviewer for Circulation, I pointed out that this wasn't a randomized trial and suggested that they soften their position. They did.

"The average physician does not truly appreciate the difference between observational and randomized clinical studies," says Weintraub. "People don't think these things are important, but it's critical to how physicians must think. There's this notion that all you need is basic science to take care of patients, and that's simply not true."

Education about epidemiology (a core discipline in public health that seeks to identify the distribution and causes of health, disease, illness, death, and injuries in populations) is sorely lacking for medical students nationwide. Emory does a better job than most medical schools of teaching the intricacies of medical research and how to use results. The School of Medicine has required a class on analytic medicine for nearly 20 years, and Emory's MD/MPH program is increasingly popular.

Weintraub says medicine must become more evidence-based. "A lot of what physicians do in practice is based on inadequate science," he says. "Physicians have a lot of experience with patients, and they derive some of their beliefs from their personal experiences. But that experience may not represent the truth. They act on their own science they've collected over time, but it's not adequate for making judgments in many cases."

When to test?

For example, doctors' opinions fall on both sides of the fence regarding prostate specific antigen (PSA) screening for prostate cancer and when to start mammography. These controversies are similar to the HRT debate. And national bodies like the American Cancer Society and the National Cancer Institute (NCI) struggle with how to position themselves.

Jack Mandel, chair of epidemiology at Emory's Rollins School of Public Health (RSPH) and an expert in cancer screening epidemiology, says PSA testing is now a subject of debate because the studies that show benefit are observational. "We haven't completed the right kind of studies to demonstrate whether it is effective," he says. "We're using data that are inadequate."

Two large, expensive, time-consuming randomized clinical trials of PSA testing are now ongoing and should yield some more definitive answers in a few years. But what should doctors do in the meantime? Some say offer the test to avoid a potential loss of life. Others argue there's no proven benefit, and the tests carry certain risks.

"Logically, you can't justify the risks if the benefit is zero," says Mandel. "Both arguments are compelling. Policymakers are stuck in the middle trying to decide what to recommend and how to position themselves. In the absence of definitive evidence, physicians often decide in discussions with individual patients what's best for them, but might not endorse mass screenings recommended by organizations. For patients, that creates a lot of confusion."

For now, the US Preventive Services Task Force does not recommend mass PSA screening, nor do the American College of Physicians and the American Academy of Family Physicians. The American Cancer Society recommends against mass screening but suggests that physicians should offer PSA testing to patients and inform them of potential risks and benefits.

Emory oncologist Otis Brawley, a prostate cancer expert and director of the Georgia Cancer Center of Excellence at Grady Memorial Hospital, says PSA testing is a question of harm versus benefit, just like HRT before the Women's Health Initiative.

At least 40% of men who are screened and then diagnosed with prostate cancer would be better off never knowing because their cancer will never cause problems. "Yet we bother them with treatment and the label 'cancer patient,'" he says. "We hope we're curing some men necessarily, but we know we're curing many men unnecessarily."

And the risks of prostate cancer treatments are substantial. Radical prostatectomy and radiation can lead to impotence, incontinence, and even death. "So patients must know that screening and aggressive treatment save lives, although treatment clearly cures some men who don't need to be cured," he says. "Until the data are in showing that PSA testing offers more good than bad, we will not have a definitive answer."

Mandel agrees. "The diagnosis and treatment of many conditions are controversial because of the controversial nature of the science behind them," he says. "Sometimes the science is not adequate to provide definitive results, and then people weigh in with opinions based on partial or incomplete scientific information. HRT and PSA testing aren't the first such issues, and they won't be the last."

Denominator gone wrong

Every seven years, John Young fills out an application for renewal of the Surveillance, Epidemiology, & End Results (SEER) program, supported by the NCI and based at the RSPH.

Established in 1973 as part of the "war on cancer," SEER continues to be the primary source of cancer statistics in the United States. Atlanta SEER data represents nearly all cancer cases in five states, four metropolitan statistical areas including Atlanta, and some rural areas, including some predominantly black Georgia counties. The CDC, American Cancer Society, NCI, and others use SEER data to analyze trends in cancer incidence and mortality, develop cancer control programs, measure the impact of prevention programs, and guide health policy decisions.

Young's application renewal strategy has traditionally been to look for trends in local cancer statistics to justify why a program absolutely must be located in Atlanta. Traditionally, Atlanta's high proportion of African Americans and racial disparities in cancer have been the plug. Atlanta has long been considered the nation's capital in terms of disproportionately high rates of cancer diagnoses and deaths among African Americans. But when Young was configuring the data for the recent application this past fall, he noticed a dramatic increase in disease. The 1999 census figures showed prostate cancer deaths for African Americans in metro Atlanta to be 182% higher than for whites, and breast cancer deaths 67% higher for African Americans.

"I was looking for a statistical twist to show why Atlanta is so important," he says. "The long-term time trends is one thing we could use. But in looking at the data, I noticed something we have never noticed before."

Since US cancer data collection first began in 1938, black men have always had higher cancer incidence rates than white men. But for women, it's been the other way around -- white women have higher rates for all cancers combined than black women. However, African Americans have higher death rates from cancer even if they have lower incidence of disease.

"Last fall, when I looked at the data for metro Atlanta, the black women had rates that were higher," he says. "And I thought: 'Oh boy, I can say we're the first to observe this, and it's really important that we continue to monitor the trends in Atlanta because other places might start to follow this trend. Then I thought before I wave this flag and say look what's happened here, I'd better be really sure. Then we did some analysis county by county, and I knew something was terribly wrong."

What went wrong was the denominator -- the total population at large. The US Census, taken every 10 years, had drastically underestimated metro Atlanta's total African-American population during the late 1990s. In the 2000 census, the numbers didn't take into account the large effect of African-American migration into metro Atlanta over the course of the decade.

Young needed accurate figures of total population to give cancer incidence and mortality numbers some sense of proportion and meaning. "When I realized that every conclusion we had come to and every rate we had calculated for 1999 was simply wrong, I was dumfounded. I couldn't believe the census bureau could have missed these people. In theory, they take school enrollment, births, and deaths. But how can a population go from 80% white to 80% black in one county and and someone not notice? The real story is with the Census Bureau and what went wrong there."

Epidemiologists depend on accurate census data to calculate incidence and mortality for all diseases. In the end, the error made the cancer disparity between blacks and whites in Atlanta - still substantial for many cancers, but now thought to be no larger than racial disparities elsewhere in the country - look much larger than it actually is.

Why is an accurate denominator so important? Whenever trying to measure cancer risk, you have to know what the population at risk actually is. Also, policy and funding decisions for cancer research are based on these figures.

Otis Brawley, an expert in racial disparities in cancer, says clearing up the confusion is essential to addressing disparities in cancer care in the state and nationwide. He needs good data to back up funding requests and to guide plans to close the racial cancer gap.

"What John Young discovered was very important, because it helped us realize that the nationally published cancer mortality data are just plain wrong!" he says. "Unfortunately, some people will take that and say there are no disparities. In reality, what John found was that all the disparities that we thought existed do exist. But the gradient in those disparities is not as bad as we thought."

Young's finding also gives Brawley ammunition in the debate over what's behind racial disparities in cancer. "There is a huge battle going on with some people saying the black-white difference is due to biologic difference, which is a total, unadulterated falsehood," he says. "The disparity in cancer death has nothing to do with racial biology. It has a great deal to do with racial differences in treatment and access to care. The numbers show that to be the case."

Scientific continuum

There are no such numbers to back many of the menopausal hormone substitutes that many menopausal women have gravitated to in the wake of the Women's Health Initiative, says Wenger.

"We know nothing about the soy products and the so-called 'natural' preparations now being marketed to women, because they're not regulated in any way. They're totally unstudied. Many women going to their local health food store receive recommendations with no evidence whatsoever. That is an important issue. Just because hormone or hormone-like preparations are natural does not mean they are safe. And if they are purported to have estrogen benefits, in theory, they may also have estrogen risks. And that is the next frontier. Will people study it? I don't know, but I have great concern about physicians who recommend it and women who use these preparations without any scientific basis. I know many physicians are nodding assent, but I certainly don't. I tell women, 'I don't know the benefits, I don't know the risks. I don't recommend anything until I know that information.'"

Weintraub says this kind of critical thinking is the wave of the future in medicine. "People think scientists discover something, and it's known for all time. But it's not that way at all. The philosophy of science is not deductive reasoning, but inductive. We learn new things, we refute old things, and we form new syntheses. It may be uncomfortable at times for us to ask questions and admit what we don't know, but that's science."

Valerie Gregg is a freelance writer.

Copyright © Emory University, 2003. All Rights Reserved.
Send comments to the Editors.
Web version by Jaime Henriquez.