Home

VRTF

Alameda County

Security Testing

Contract Problems

Testing Rules

Whitewash

Background

On the June 8th, 2006 meeting of the Alameda County Board of Supervisors, an amendment to the Sequoia contract was approved for security and vulnerability testing. The intention of the BoS was evident during the discussion that led up to the vote. The term "hack test" was used repeatedly. The most telling quote is the final amendment statement by Supervisor Lai Bitker and commented on by Supervisor Carson.

Hack testing is a slang/colloquial term with no formal definition in the computer security community. Similar terms with more specific meanings are red team attack and penetration test which mean assessing and testing vulnerabilities and limitations of systems or structures from an adversarial perspective. For the purposes of clear definition, for the rest of this document "hack test" won't be used. Instead, security evaluation test will be used and the definition will include an exploration and learning phase where the evaluator works to understand the machine and a second phase of "attack demonstration" showing what could be done in a real election.

For a test of this type to be truly meaningful it must be free from any manipulation of the results. This would include, but not be limited to, independent selection of equipment to test and verification that the software is the same as the SoS has on file. For a full assessment to be truly meaningful it should include, but not be limited to, access to source code and other documentation.

A security and vulnerability test, as voted on the BoS, needs to include a security evaluation test as a key subset to that testing. Assessment of the whole system must include this testing as part of the system testing for the testing to be meaningful.

The citizens of Alameda County insist that the Board of Supervisors and their Staff, including the Registrar of Voters, the General Services Administration, and the County Counsel uphold the amendment and it's intent prior to the November 7th election. The testing and assessment must be done in a timely manner such that the County and candidates for office can have time to take action based on the results.

Once the testing is done, the BoS is to discuss in closed session how to make the results public. Examples of public disclosure that do not compromise the security of the system include the VSTAAB report on Diebold.

Please see back up in Appendix A to all statements in this BACKGROUND section.

Summary

Security vulnerability testing can only demonstrate that a system is vulnerable. It cannot demonstrate that a system is not vulnerable. This is similar to testing the passenger screening procedures at an airport. If a tester can get through the screening carrying a weapon, it proves that the system is vulnerable. If the tester is stopped, it does not prove that the system is invulnerable, only that one tester was stopped. Security vulnerability testing also does not address issues such as hidden code (Easter Eggs) present in the software or firmware.

It would be very much in the public's interest for the county to conduct a thorough security analysis, including independent:

• inspection of the source code (such as San Mateo county intends to conduct),

• inspection of the hardware inside the casing including the motherboard,

• review of high and low level hardware design documents,

• inspection and analysis of the firmware,

• inspection of ballot definition files, and all other data files,

• review of procedures,

• testing where deemed necessary by the analysis team.

"Security Evaluation Test" Goals

• Can an outsider or insider cause they system to incorrectly record a vote?

• Can an outsider or insider change the vote count to incorrect totals?

Testers

• As stated in the vote of June 8, the testers need to be independent, which means they must not be employees of Sequoia nor members of the county staff that have now become "invested" in the proposed system.

• The testers should be persons with unquestioned competence and experience in computer security.

System Hardware, Software, And Firmware To Be Tested

• All equipment must be the actual equipment that the county will use.

• The vendor's software being installed must be the Secretary of State's trusted build used in the certification testing. The Secretary of State's office will need to provide whatever is necessary for installing the software.

• The testers will reformat the hard drive(s) of, and install the exact same software on the test tabulator as well as on the tabulator to be used on election day.

• Where COTS software is involved, the testers will install it on the test machines. The county or the vendor will supply the software.

• The Secretary of State's trusted software will be installed by the testers on all 4 central scanners and the 5 memory pak readers.

• Key attack point areas were NOT tested by ITA testers and should be tested by the county. See Appendix B Brennan Center report, software attacks section.

Test Protocols

• Any part of the whole system may be tested.

• DREs and precinct scanners will be randomly selected from those delivered by Sequoia.

• Testers may use their own test plans, not any plans provided by the Vendor.

• The testing team(s) may make multiple attempts at different parts of the system.

• The testers will know the version of the tabulator's SQL data base management system at least two weeks before the start of testing.

• The testers may use any equipment they wish to, including, but not limited to, laptops, card readers, EEPROM readers, etc.

• The testers may use video, photographic, and audio equipment and take notes.

• Any installation of any software by anybody may be observed by anybody.

Timing

• The testing shall take place before payment is made.

• The testing shall take place before the elections in November, 2006.

• The testers may set the date on the computer equipment to election day.

• The testers shall have a minimum of 2 weeks with the machines.

Publication

• There will be 2 reports. One, complete report by the testing team, to be delivered to the vendor, the county, the SoS, and other relevant, responsible government officials at all levels, ASAP.

• Redacted reports shall be released to the public ASAP. Redactions should be kept to a minimum, and redactions should be permitted only to (a) protect the legitimate intellectual property of the vendor, or to (b) conceal any information whose disclosure could compromise election security.

• The unredacted report should be published at the latest 1 month before the next certification hearing of the system, so that the public has an opportunity to make informed comments. The testing team may make recommendations for earlier publication of certain sections, depending on the severity of the vulnerability, and the risks involved.

Observers

• The testing should be public, if possible. If not, in accordance with section 15004 of the California Elections Code (below), each political party may have present two qualified observers.

• Observers may use video, photographic, and audio equipment and take notes.

15004, Technical Observers

California Elections Code, Section 15004 : "The county central committee of each qualified political party may employ, and may have present at the central counting place or places, not more than two qualified data processing specialists or engineers to check and review the preparation and operation of the tabulating devices, their programming and testing, and have the specialists or engineers in attendance at any or all phases of the election."

Nobody, and no machine, should be counting American votes in secret.