Zogby poll of US service members in Iraq - Page 3

mtnbiker · 03-03-2006, 15:51 PM

gunnut/Snipe,

thanks for posting that.

gunnut · 03-03-2006, 18:06 PM

You're very welcome.

Snipe used to work for the government and he's here to help

**Shek** · 03-03-2006, 19:29 PM

Well, peer review problems are starting to emerge on the poll, further casting doubt on its validity.

Quote:

http://www.mysterypollster.com/main/...oop_pol_2.html

March 03, 2006
Zogby Troop Poll: The Random Probability Sample
On the Zogby poll of U.S. troops in Iraq, I need to make one point that was implicit in my comments on Wednesday a bit more explicit. While much is shrouded in secrecy, one aspect of the methodology is clear from the information that John Zogby has provided on-the-record: The survey did not involve a "random probability" sample of all American troops serving in Iraq.

The principle of random sampling is what makes a poll "scientific." To meet that standard in this case, every member of the U.S. armed services in Iraq should have had some chance of being selected (or to put it statistical terms, the probability of selection had to be either equal or known for every member of the population). As I wrote yesterday, the constraints Zogby faced in gaining access to troops at "undisclosed locations throughout Iraq" made random selection of those locations impossible.

It is also unclear -- both from information in the public domain and from what Zogby shared with me in confidence -- whether his selection procedures amounted to random probability sampling even at the undisclosed locations. I did not press Zogby for details on that process, because under the terms of our agreement I could not report the details. While I could speculate about his procedures, unfortunately, doing so would require disclosure of information I promised not to disclose.

However, as this example provides an opportunity to learn something about the survey process, consider how the Gallup organization went about conducting a "strict probability based" random sample of ordinary Iraqis citizens in 2004. To grossly oversimplify the design, Gallup randomly selected 350 neighborhoods ("primary sampling units") from a list of over 116,314 in Iraq (the "sample frame"), using population statistics to make sure the probability of selection was proportional to the size of the neighborhood. In other words, every neighborhood had an equal probability of being selected.

Then, interviewers went to each neighborhood and compiled a list of every family living within every dwelling in that neighborhood and randomly selected families from that list. Within each selected household that agreed to participate, they took an inventory of all family members over 18 years of age and randomly selected one adult to be interviewed in a way that insured that both genders had an equal chance of inclusion, with no one allowed to self-select into the sample.

Thus, Gallup used a standardized procedure that gave every Iraqi adult (excluding those in institutions and those in two small Kurdish "governates" - see the footnote) a chance of selection, and the selection procedure was random at every step. That, as Gallup puts it, is a "strict, probability based sample design." Note also how much detail Gallup was willing to disclose about their sampling methods, despite very real concerns about the safety of their interviewers.

Now consider one of the examples I mentioned on Wednesday, the survey of Katrina evacuees conducted jointly last year by the Washington Post, the Kaiser Family Foundation and the Harvard School of Public Health. In that case, it was not feasible to use random techniques to try to survey the population of all of those who had to evacuate their homes because of Katrina. The full population had scattered widely, and the researchers lacked a public list of all evacuees or count of evacuees by their new geographic locations. So instead they did a survey limited to some (but not all) of the shelters housing evacuees in the Houston area. The selection of these locations was neither random nor representative of all evacuees, but the pollsters made no such claims. Instead they were careful to report the results as projective of "evacuees living in shelters" in Houston.

But review the procedures the researchers used and you will see the effort made to randomly select respondents at each location.

Quote:

For areas where the evacuees either had limited mobility or were non-mobile -- --for example, cot areas occupied largely by elderly or infirm evacuees, or TV lounge areas -- interviewers moved through the respondent population. Specifically, interviewers were given a random number and instructed to count off this number of people before beginning the first/next interview. After an interview was completed (or a refusal obtained), interviewers would again count off using the random interval before selecting the next respondent.

For areas where evacuees were mobile -- for example hallways and evacuee service areas -- interviewers stayed in one particular spot throughout the interviewing period. They then counted people who passed their defined location and chose the (randomly generated) nth person to interview. This selection criteria was duplicated at the conclusion of each contact attempt, whether it was a completed interview or a refusal.

So what can we say about the degree to which the Zogby survey used random probability sampling to survey U.S. troops? Again, as I wrote earlier in the week, the method Zogby used to gain access to the undisclosed locations constrained his ability to select them. The selection was not random, but since he will not disclose the locations, we cannot take their identity into account in evaluating the results. The release specifies a sampling error of 3.3% (a statistic that, given the sample size, is based on the assumption of simple random sampling), but that margin is a bit deceiving. Plus or minus 3.3% compared to what? All we know for certain is that the poll was not a random sample of the population of all U.S. troops in Iraq.

As to the selection of respondents at those unspecified locations, we also do not know the procedures used to select respondents. Again, I did not press Zogby on the details of those procedures in our conversation because I would not have been able to report them here. I believe MPs readers deserve more than "trust me" as an explanation. Obviously, for me to speculate now about what Zogby might have done would require getting into the details I promised I would not reveal.

Some news organizations (like ABC News) have adopted strict standards that require the use of probability sampling and bar reporting of surveys based on intercept selection techniques in absence of a "credible sampling frame." Others are obviously less rigorous.

In the business world, commercial market researchers sometimes use non-random sampling (including many Internet based "panel" surveys) when rigorous probability samples are impractical or prohibitively expensive. However, the most ethical of these market researchers do not attempt to dress up such "convenience" samples as more than they are. Their clients pay for such projects on the assumption that the information obtained, while imperfect, is the best available.

John Zogby insists it is enough that those of us who have heard more about his survey's methodology conclude that it was "honestly and objectively done." I think he misses an important point. Consumers of Zogby's Iraq troop poll data also need to understand where it fits on the continuum between strict probability-based sampling and non-random convenience sampling. Zogby certainly believes that "security concerns" prevent further disclosure, that we do not "need to know" more. Perhaps. But without knowing more, it is hard to decide whether to trust the results.

Bluesman · 03-10-2006, 19:27 PM

More on the poll:

Quote:

March 10, 2006
Zogby's Bogus Poll on U.S. Military in Iraq
By Tim Kane

For anyone following the Iraq war, now may be the time to take off those rose-colored glasses. According to a recent Zogby poll, 72 percent of U.S. troops say it's time to withdraw from Iraq. Another stunner is that only three in five soldiers in Iraq have a clear sense of the mission. Ouch!

Despite the wide range of opinions and loud voices about America's role in Iraq, there's a real hunger for authenticity that only the troops on the ground can provide. As a veteran, I have been hoping that a pollster would take the obvious step of asking our troops for their opinions, and I think Zogby International deserves credit for making the effort.

But as an economist, my appreciation eroded sharply when I took a closer look.

The survey contains 24 questions. It was given secretly during late January and early February to an unknown number of American troops serving in Iraq, although we are told that 944 respondents were included. If all the guidelines for random sampling were met (they weren't), the reported margin of error would be plus or minus 3.3 percent.

The unforgivable flaw in Zogby's survey is the biased phrasing of its questions and answers. Two of the most provocative results are based on questions with no middle ground. It's like a multiple-choice test with no right answers.

For example, the widespread finding that three in four soldiers think the United States should withdraw from Iraq within a year has only one option for troops who think otherwise: stay indefinitely. This infamous question asks, "How long should U.S. troops stay in Iraq?" But the first three answers are not phrased in terms of staying, they are phrased "withdraw...," "withdraw..." and "withdraw... ." Where are the options for troops who think the United States should stay for "one to two years" or "two to five years"? Zogby omits such nuance. It's stay or go. Now or never.

The smart troops who perceived this false choice probably set the clipboard down and walked away at that point. That leaves us with a biased respondent pool.

Another question asks for a description of "your understanding of the U.S. mission in Iraq." Two choices describe the mission as clear, and four choices describe it as unclear.

More damning, John Zogby himself misrepresented the phrasing of one of the questions in an op-ed. This may seem like nitpicking, but if half a man's family say they want "chicken" for dinner, and he reports those votes as "nonvegetarian," he is not exactly being honest. In just this way, the poll asked the soldiers to rate seven different "reasons for the Iraq invasion." It is a question about prewar justification, not the postwar occupation. Yet Zogby described their answers as a description of "the U.S. mission." If that's the question he wanted to ask, he should have asked it that way. Polling is a science. Words matter.

The biggest question we should all be asking Zogby is not about the questions that were included, but about those that weren't. Nowhere in the survey results do we see assessments of the U.S. mission. Has it been a success or a failure? How so? Nowhere do we see questions about morale, about progress in killing terrorists, about the state of the insurgency, about the prospects for democracy and economic growth in Iraq. There are questions aplenty on napalm, interrogation, and (I'm not kidding) doubling the number of bombing missions.

Did Zogby dare to ask anything that might result in good news?

Keep in mind that the men and women in uniform are limited by law from making political statements. If troops are given a chance to express themselves anonymously and fairly, that's great. They are probably the best barometer of how the mission is going, and how it can be improved. But this Zogby poll isn't a barometer. It is (a) biased, (b) dishonest, or (c) all of the above.

Don't like those options? Neither do I. But that's all they gave us.

I remain thankful that Zogby made this effort, and I hope they will try again in a manner that is (d) insightful, (e) comprehensive, and maybe even (f) irrefutably profound.

Tim Kane (Tim.Kane@heritage.org) is an economist and Bradley Fellow in the Center for Data Analysis at the Heritage Foundation, and a veteran Air Force officer.

2DREZQ · 03-11-2006, 20:58 PM

Not to wander off-topic here, but...

"Four in five said they oppose the use of such internationally banned weapons as napalm and white phosphorous. "

Did somebody ban something without telling me?

Jeez, I hate it when I get left out of the loop!

**Shek** · 03-11-2006, 22:20 PM

Quote:

Originally Posted by I'm Back!

Not to wander off-topic here, but...

"Four in five said they oppose the use of such internationally banned weapons as napalm and white phosphorous. "

Did somebody ban something without telling me?

Jeez, I hate it when I get left out of the loop!

They aren't banned, but their use is restricted.

Ray · 03-11-2006, 23:19 PM

One should not be surprised at the results of the poll.

It would have been similar for any army in the world in such a situation.

War or assymetrical warfare is OK in the beginning, but then it gets tiresome and yet one still has to stick on for the security of the nation, or, for the credibility of the nation's mission.

Whatever maybe the soldiers' or the reservists' inner conviction, it is always on his/ her subconscious that if he/ she did not do what they are doing, then the nation would suffer and possibly be humiliated! That is a very powerful sentiment and that is in all soldiers, thanks to their training or otherwise.

No soldier ever willing is ready to let the nation down and yet most soldiers are not.

While most soldiers would like the combat situation to end if possible immediately, yet most soldiers would not hesitate to hang on forever if it was to the interest of their nation.

The fact that India has been combatting such a situation over two plus decades without let up would indicate that soldiers are made of sterner stuff and their loyalty to the nation is unquestionable, be they of the US or India.

I don't think it is a fudged poll.

**Shek** · 03-11-2006, 23:29 PM

Quote:

Originally Posted by Ray

One should not be surprised at the results of the poll.

It would have been similar for any army in the world in such a situation.

War or assymetrical warfare is OK in the beginning, but then it gets tiresome and yet one still has to stick on for the security of the nation, or, for the credibility of the nation's mission.

Whatever maybe the soldiers' or the reservists' inner conviction, it is always on his/ her subconscious that if he/ she did not do what they are doing, then the nation would suffer and possibly be humiliated! That is a very powerful sentiment and that is in all soldiers, thanks to their training or otherwise.

No soldier ever willing is ready to let the nation down and yet most soldiers are not.

While most soldiers would like the combat situation to end if possible immediately, yet most soldiers would not hesitate to hang on forever if it was to the interest of their nation.

The fact that India has been combatting such a situation over two plus decades without let up would indicate that soldiers are made of sterner stuff and their loyalty to the nation is unquestionable, be they of the US or India.

I don't think it is a fudged poll.

Sir,
The science is bad, the wording of some questions is questionable and other questions downright loaded, and while I wouldn't be surprised at a mixed response on future Iraq policy questions, this poll is bad in that it doesn't even explore any linkage to the responses in this area. Thus, you can't really decipher anything from these numbers, even if they were completely valid.

Confed999 · 03-11-2006, 23:30 PM

Quote:

Originally Posted by Ray

One should not be surprised at the results of the poll.

I am, for the simple reason I haven't heard those things from the returning soldiers. Granted I've only talked to maybe a dozen, plus shek, but that is not what they're saying. After reading the articles providing more info on the poll, I'm not surprised I was surprised.

Ray · 03-11-2006, 23:43 PM

While I am not contesting any of your opinions per se, but all polls have their statistical probability of error and that is compounded by poor question wording or motivated wordings etc.

I was merely stating that very few love war as a pastime, and an endless one at that.

Counter Insurgency is an assymetrical war with the end of the tunnel too far for visual discernment.

Yet, if the chips are down, no matter what maybe their personal feelings, a soldier will never quit!

**Shek** · 03-11-2006, 23:56 PM

Quote:

Originally Posted by Ray

While I am not contesting any of your opinions per se, but all polls have their statistical probability of error and that is compounded by poor question wording or motivated wordings etc.

I was merely stating that very few love war as a pastime, and an endless one at that.

Counter Insurgency is an assymetrical war with the end of the tunnel too far for visual discernment.

Yet, if the chips are down, no matter what maybe their personal feelings, a soldier will never quit!

Sir,

I agree, which is why I wouldn't be surprised with a mix of responses, although not to the degree that this non-random sample has. However, a well designed poll would have followed the withdrawal timeline with some motivating questions about their answer to the timeline question: are you tired of Iraq rotations? do you think the US will suceed or fail? etc. Instead, this poll leaves that question open and allows for maneuver room for commentators to imply reasons.

Additionally, the poll is not representative of the US military population in Iraq. Thus, this introduces its own bias and imprecision, and results cannot be extrapolated to the population - to do so violates the very science of statistics upon which polling relies on.

Anon · 03-12-2006, 00:51 AM

Quote:

Originally Posted by shek

They aren't banned, but their use is restricted.

WP is illegal for use against EPERS so far as i know. GG has confirmed that in another discussion he and i had about marking targets for "smoke" rounds.

**Shek** · 03-12-2006, 08:06 AM

Quote:

Originally Posted by M21Sniper

WP is illegal for use against EPERS so far as i know. GG has confirmed that in another discussion he and i had about marking targets for "smoke" rounds.

No, it's legal against enemy personnel. It's illegal to use in densely populated areas in order to protect civilians.

http://www.globalsecurity.org/milita...s_prot-iii.htm

I discussed this ad naseum back in November/December when knuckleheads where parroting an Italian document that showed decomposed bodies where the clothing was still intact (wow, the thousands of degree temperature of WP is miraculously retarded by the all natural fibers worn by Iraqis

), which was claimed to be proof of a deliberate use of WP against civilians in Fallujah.

Anon · 03-12-2006, 11:38 AM

Hmm, must've been one of those milmyths, like "You cant shoot people with .50cal, it's illegal".

That aint illegal either...

03-11-2006, 23:19 PM	#37 (permalink)
Ray Postmaster General *Military Professional* Join Date: 08-20-03 Posts: 27,195 Country:	One should not be surprised at the results of the poll. It would have been similar for any army in the world in such a situation. War or assymetrical warfare is OK in the beginning, but then it gets tiresome and yet one still has to stick on for the security of the nation, or, for the credibility of the nation's mission. Whatever maybe the soldiers' or the reservists' inner conviction, it is always on his/ her subconscious that if he/ she did not do what they are doing, then the nation would suffer and possibly be humiliated! That is a very powerful sentiment and that is in all soldiers, thanks to their training or otherwise. No soldier ever willing is ready to let the nation down and yet most soldiers are not. While most soldiers would like the combat situation to end if possible immediately, yet most soldiers would not hesitate to hang on forever if it was to the interest of their nation. The fact that India has been combatting such a situation over two plus decades without let up would indicate that soldiers are made of sterner stuff and their loyalty to the nation is unquestionable, be they of the US or India. I don't think it is a fudged poll. __________________ "Some have learnt many Tricks of sly Evasion, Instead of Truth they use Equivocation, And eke it out with mental Reservation, Which is to good Men an Abomination." I don't have to attend every argument I'm invited to. HAKUNA MATATA

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Kerry Slams U.S. Foreign Policy	Enzo Ferrari	Political Discussions	252	02-08-2007 05:34 AM
A moment of silence please...	troung	The War in Iraq	0	06-09-2006 17:19 PM
25,000 civilians KilledIn Iraq	jimmy22	The War in Iraq	88	08-13-2005 11:41 AM
India edges back into Iraq	Srirangan	International Defense Topics	0	05-30-2005 02:32 AM
More proof of the Saddam al Qaeda Connection	Leader	International Defense Topics	3	05-31-2004 21:22 PM

03-03-2006, 15:51 PM	#31 (permalink)
mtnbiker Patron Join Date: 02-16-04 Location: California Posts: 191	gunnut/Snipe, thanks for posting that.

03-03-2006, 18:06 PM	#32 (permalink)
gunnut *Senior Contributor* Join Date: 01-27-06 Location: DPRK, Democratik People's Republik of Kalifornia Posts: 9,867 Country:	You're very welcome. Snipe used to work for the government and he's here to help

03-11-2006, 20:58 PM	#35 (permalink)
2DREZQ Contributor Join Date: 08-04-03 Posts: 704	Not to wander off-topic here, but... "Four in five said they oppose the use of such internationally banned weapons as napalm and white phosphorous. " Did somebody ban something without telling me? Jeez, I hate it when I get left out of the loop!

03-11-2006, 23:43 PM	#40 (permalink)
Ray Postmaster General *Military Professional* Join Date: 08-20-03 Posts: 27,195 Country:	While I am not contesting any of your opinions per se, but all polls have their statistical probability of error and that is compounded by poor question wording or motivated wordings etc. I was merely stating that very few love war as a pastime, and an endless one at that. Counter Insurgency is an assymetrical war with the end of the tunnel too far for visual discernment. Yet, if the chips are down, no matter what maybe their personal feelings, a soldier will never quit!

03-12-2006, 11:38 AM	#44 (permalink)
Anon New Member Join Date: 08-03-03 Posts: 0	Hmm, must've been one of those milmyths, like "You cant shoot people with .50cal, it's illegal". That aint illegal either...

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)