CovOps
 Location : Ether-Sphere
Job/hobbies : Irrationality Exterminator
Humor : Über Serious
 |  Subject: New anti-terror weapon: Hand-held lie detector  Fri Apr 11, 2008 9:05 am | |
| FORT JACKSON, S.C. - The Pentagon will issue hand-held lie detectors this month to U.S. Army soldiers in Afghanistan, pushing to the battlefront a century-old debate over the accuracy of the polygraph.
The Defense Department says the portable device isn't perfect, but is accurate enough to save American lives by screening local police officers, interpreters and allied forces for access to U.S. military bases, and by helping narrow the list of suspects after a roadside bombing. The device has already been tried in Iraq and is expected to be deployed there as well. “We're not promising perfection — we've been very careful in that,” said Donald Krapohl, special assistant to the director at the Defense Academy for Credibility Assessment, the midwife for the new device. “What we are promising is that, if it's properly used, it will improve over what they are currently doing.”
But the lead author of a national study of the polygraph says that American military men and women will be put at risk by an untested technology. "I don't understand how anybody could think that this is ready for deployment," said statistics professor Stephen E. Fienberg, who headed a 2003 study by the National Academy of Sciences that found insufficient scientific evidence to support using polygraphs for national security. "Sending these instruments into the field in Iraq and Afghanistan without serious scientific assessment, and for use by untrained personnel, is a mockery of what we advocated in our report."
The new device, known by the acronym PCASS, for Preliminary Credibility Assessment Screening System, uses a commercial TDS Ranger hand-held personal digital assistant with three wires connected to sensors attached to the hand. An interpreter will ask a series of 20 or so questions in Persian, Arabic or Pashto: "Do you intend to answer my questions truthfully?" "Are the lights on in this room" "Are you a member of the Taliban?" The operator will punch in each answer and, after a delay of a minute or so for processing, the screen will display the results: "Green," if it thinks the person has told the truth, "Red" for deception, and "Yellow" if it can't decide.
The PCASS cannot be used on U.S. personnel, according to a memo authorizing its use, signed in October by the undersecretary of defense for intelligence, James R. Clapper Jr.
A model of the PCASS, with sensors on the palm instead of the fingertips.
The Army has bought 94 of the $7,500 PCASS machines, which are sold by Lafayette Instrument Co. of Lafayette, Ind. The algorithm, or computer program that makes the decisions, was written by the Advanced Physics Lab at Johns Hopkins University. Besides the Army, other branches of the U.S. military have seen the device and may order their own. The total cost of the project so far is about $2.5 million.
Congress has not held any hearings on the PCASS device. Myron Young, a spokesman for the Pentagon’s Counterintelligence Field Activity agency, which sponsored development of PCASS, said it informed congressional committees in a memo in November that the device had been approved for use. But five months later, no hearing has been scheduled. Congress has already scaled back its oversight of the polygraph. Five years ago it eliminated a requirement that the Defense Department produce an annual report on polygraph use.
Less accurate than a polygraph
Polygraphs have sparked a fierce debate for at least a century. While supporters claim the devices are reliable, the National Academy of Sciences allows only that they're "well above chance, though well below perfection." Polygraphs are not allowed as evidence in most U.S. courts, but they're routinely used in police investigations, and the Defense Department relies heavily on them for security screening.
Both the proponents and critics agree on one thing: This new device is likely to be less accurate than a polygraph, because it gathers less physiological information.
A PCASS screen displays "Red," indicating the device has decided the subject is being deceptive. The screen also shows the question that prompted the strongest response, in this case regarding a bomb threat.
Like a polygraph, the PCASS uses two electrodes to attempt to measure stress through changes in electrical conductivity of the skin. It also gauges cardiovascular activity, though with a pulse oximeter clipped to a fingertip, rather than a polygraph's arm cuff, which has the advantage of measuring both pulse rate and blood pressure. Unlike the polygraph, the PCASS does not measure changes in the rate of breathing, and it has no way to detect countermeasures, or efforts to fool the machine, such as by making unusual movements.
The training is different, too. While polygraph examiners for the Defense Department must have four-year college degrees and experience in law enforcement, the PCASS operators are mostly mid-level enlisted personnel and warrant officers, some as young as 20 years old. While polygraph examiners take a 13-week course and a six-month internship, PCASS operators undergo only one week of training, though most have military training as interrogators. The Defense Department says PCASS is simple to operate, because judgment of truthfulness is left to the computer.
Discarding 'inconclusives'
The debate over the device’s usefulness boils down to a disagreement over its accuracy.
The Pentagon, in a PowerPoint presentation released to msnbc.com through a Freedom of Information Act request, says the PCASS is 82 to 90 percent accurate. Those are the only accuracy numbers that were sent up the chain of command at the Pentagon before the device was approved.
But Pentagon studies obtained by msnbc.com show a more complicated picture: In calculating its accuracy, the scientists conducting the tests discarded the yellow screens, or inconclusive readings.
BACKGROUNDER
How accurate is it?
The Defense Department has paid for three studies of the handheld lie detector, PCASS. Here are brief descriptions. Follow the links to read the full study reports.
Test one
In the first test, Army basic trainees at Fort Jackson, S.C., were put through what's called a mock-crime drill. Some recruits were told to place a fake bomb by a roadside, while others remained inside. After a detonation was heard, each was questioned.
They were given an incentive to tell the truth: If they were found to be deceptive, they would have to give a 10-minute speech to their unit on the subject of honesty. (That threat was itself a lie.) The examiners were all polygraph instructors, and the interviews were in English.
The test was performed in three rounds: first with a traditional polygraph, then twice with PCASS.
In the first round, with a polygraph, the examiner was correct 79.7 percent of the time, or 55 out of 69. The Pentagon prefers to cite an accuracy rate that sets aside the "yellow" results, or inconclusives; that yields a success rate of 85.9 percent, or 55 out of 64. (The polygraph was uncertain in 5 cases out of 69, or 7.2 percent.)
In the second round, with PCASS, the device was correct in 63.0 percent of the cases, or 46 out of 73. The Pentagon cites an accuracy rate, setting aside the inconclusives, of 86.8 percent, or 46 out of 53. (The PCASS was uncertain in 20 out of 73 cases, or 27 percent.)
In the third round, with PCASS, the device was correct in 62.2 percent of the cases, or 51 out of 82. The Pentagon cites an accuracy rate, setting aside the inconclusives, of 73.9 percent, or 51 out of 69. (The PCASS was uncertain in 13 out of 82 cases, or 16 percent.)
Test two
In the second test, in Columbus, Ohio, civilians who answered a classified ad for a scientific study on deception. Their incentive: Participants were paid a $50 bonus if the machine showed them to be truthful. Battelle Memorial Institute, a defense contractor, was paid $305,000 to perform this test. The instructors were experienced law-enforcement polygraphers, and the participants were all American, English-speaking and college-educated.
Again, this was a mock-crime test, with the fake theft of a ring from a secretary's desk. Both those who did and didn't steal the ring were then questioned.
The PCASS was correct in 78.9 percent of the cases, or 56 out of 71. Setting aside the inconclusive, the Pentagon cites an accuracy rate of 91.8 percent, or 56 out of 61. (The PCASS was uncertain in 10 out of 71 cases, or 14 percent.)
Test three
In the third test, the algorithm that makes the decisions was tested in the lab by its creators at the Johns Hopkins University Advanced Physics Lab. The university was paid $1.2 million for its work.
The PCASS was still being developed while they wrote the software, so they had no PCASS exam data to work with. They used a set of polygraph exam records, part of the same set of records that were used to develop the device. All of these records had been independently verified, such as using urinalysis to prove whether or not someone was lying about using drugs.
For every 100 deceptive people, the researchers reported, the device would detect 86 (red), with two false negatives (green) and 12 uncertain (yellow).
For every 100 truthful people, they said, it would detect 50 (green), with eight false positives (red) and 42 uncertain (yellow).
That practice was criticized in the 2003 National Academy study, which said the "inconclusives" have to be included to measure accuracy. If you take into account the yellow screens, the PCASS accuracy rate in the three Pentagon-funded tests drops to the level of 63 to 79 percent.
Even if you accept the lower accuracy rates, the Pentagon officials say, the device is still better than relying on human intuition.
"Let's take a worst-case scenario here, and let's say PCASS really is 60 percent accurate," said Krapohl, who heads the project for the Defense Academy for Credibility Assessment at Fort Jackson, S.C. "So let's get rid of the PCASS because it makes errors, and go back to the approach we're currently using, which has less accuracy? As you can see, that's really quite untenable if we're interested in saving American lives and serving the interests of our commanders overseas.”
Msnbc.com asked Fienberg, a professor of statistics and social sciences at Carnegie Mellon University in Pittsburgh, to review the unclassified Pentagon studies of the PCASS. He said he was not impressed.
"I, like everyone else I know, want the troops in Iraq, in Afghanistan, elsewhere in the world, to be protected," Fienberg said. "I want terrorists to be detected. I do not want to be living in a threatened world, and I want to give my government the best possible advice.
"They need devices that work. And if they rely on things that really don't work, and act as if they do, we will have a greater disaster on our hands than we already do in the field in Iraq. We simply do not know what a device like this hand-held device will produce in that kind of setting, except for the fact that there's scant evidence that it will produce anything of value."
BACKGROUNDER
How accurate is it?
The Defense Department has paid for three studies of the handheld lie detector, PCASS. Here are brief descriptions. Follow the links to read the full study reports.
Test one
In the first test, Army basic trainees at Fort Jackson, S.C., were put through what's called a mock-crime drill. Some recruits were told to place a fake bomb by a roadside, while others remained inside. After a detonation was heard, each was questioned.
They were given an incentive to tell the truth: If they were found to be deceptive, they would have to give a 10-minute speech to their unit on the subject of honesty. (That threat was itself a lie.) The examiners were all polygraph instructors, and the interviews were in English.
The test was performed in three rounds: first with a traditional polygraph, then twice with PCASS.
In the first round, with a polygraph, the examiner was correct 79.7 percent of the time, or 55 out of 69. The Pentagon prefers to cite an accuracy rate that sets aside the "yellow" results, or inconclusives; that yields a success rate of 85.9 percent, or 55 out of 64. (The polygraph was uncertain in 5 cases out of 69, or 7.2 percent.)
In the second round, with PCASS, the device was correct in 63.0 percent of the cases, or 46 out of 73. The Pentagon cites an accuracy rate, setting aside the inconclusives, of 86.8 percent, or 46 out of 53. (The PCASS was uncertain in 20 out of 73 cases, or 27 percent.)
In the third round, with PCASS, the device was correct in 62.2 percent of the cases, or 51 out of 82. The Pentagon cites an accuracy rate, setting aside the inconclusives, of 73.9 percent, or 51 out of 69. (The PCASS was uncertain in 13 out of 82 cases, or 16 percent.)
Test two
In the second test, in Columbus, Ohio, civilians who answered a classified ad for a scientific study on deception. Their incentive: Participants were paid a $50 bonus if the machine showed them to be truthful. Battelle Memorial Institute, a defense contractor, was paid $305,000 to perform this test. The instructors were experienced law-enforcement polygraphers, and the participants were all American, English-speaking and college-educated.
Again, this was a mock-crime test, with the fake theft of a ring from a secretary's desk. Both those who did and didn't steal the ring were then questioned.
The PCASS was correct in 78.9 percent of the cases, or 56 out of 71. Setting aside the inconclusive, the Pentagon cites an accuracy rate of 91.8 percent, or 56 out of 61. (The PCASS was uncertain in 10 out of 71 cases, or 14 percent.)
Test three
In the third test, the algorithm that makes the decisions was tested in the lab by its creators at the Johns Hopkins University Advanced Physics Lab. The university was paid $1.2 million for its work.
The PCASS was still being developed while they wrote the software, so they had no PCASS exam data to work with. They used a set of polygraph exam records, part of the same set of records that were used to develop the device. All of these records had been independently verified, such as using urinalysis to prove whether or not someone was lying about using drugs.
For every 100 deceptive people, the researchers reported, the device would detect 86 (red), with two false negatives (green) and 12 uncertain (yellow).
For every 100 truthful people, they said, it would detect 50 (green), with eight false positives (red) and 42 uncertain (yellow).
http://www.msnbc.msn.com/id/23926278 |
|
