Secretary of State Debra Bowen conducted a top-to-bottom review in 2007 of many of the voting systems certified for use in California. The review, led by computer scientists from the University of California, was designed to restore the public's confidence in the integrity of the electoral process and to ensure that California voters cast their ballots on machines that are secure, accurate, reliable, and accessible. Following the top-to-bottom review, on August 3, 2007, Secretary Bowen strengthened the security requirements and use conditions for certain systems. The following documents detail Secretary Bowen's decisions and the independent experts' findings in the review.
Withdrawal of Approval and Reapproval Decisions Issued by Secretary of State Debra Bowen
Premier Election Solutions (formerly Diebold Election Systems)
- Withdrawal of Approval/Conditional Reapproval - October 25, 2007 Revision (PDF)
- Withdrawal of Approval/Conditional Reapproval - October 25, 2007 Redline Version (PDF)
- Post-Election Manual Tally Regulations [expired]
Hart InterCivic
- Withdrawal of Approval/Conditional Reapproval - December 6, 2007 Revision (PDF)
- Withdrawal of Approval/Conditional Reapproval - December 6, 2007 Red Line Version (PDF)
- Post-Election Manual Tally Regulations [expired]
Sequoia Voting Systems
- Withdrawal of Approval/Conditional Reapproval - October 1, 2009 Revision (PDF)
- Withdrawal of Approval/Conditional Reapproval - October 1, 2009 Redline Version (PDF)
- Withdrawal of Approval/Conditional Reapproval - October 25, 2007 Revision (PDF)
- Withdrawal of Approval/Conditional Reapproval - October 25, 2007 Redline Version (PDF)
- Post-Election Manual Tally Regulations [expired]
Election Systems and Software
- ES&S Inkavote Plus Conditional Approval - January 2, 2008 (PDF)
- InkaVote Plus Source Code Report - October 2, 2007 (PDF)
- InkaVote Plus Red Team Report - October 2, 2007 (PDF)
- InkaVote Plus Voting System Accessibility Review - January 2, 2008 (PDF)
- InkaVote Plus - November 26, 2007 Public Hearing Notice (PDF)
- InkaVote Plus - November 26, 2007 Hearing Transcript (PDF)
- Rescission and Withdrawal of Approval - August 3, 2007 (PDF)
- Post-Election Manual Tally Regulations [expired]
UC Final Reports
Following are the final reports from the University of California scientists detailing their findings from the top-to-bottom review. The red, source code, and documentation review team reports are separated by voting system. The accessibility report contains findings on all of the voting systems that were reviewed.
UC Source Code Team Reports
- UC Principal Investigator David Wagner's Statement on Protection of Security-Sensitive Information (PDF)
- Premier Election Solutions (formerly Diebold) (PDF)
- Hart InterCivic (PDF)
- Sequoia Voting Systems (PDF)
UC Red Team Reports
- Overview by UC Principal Investigator Matt Bishop (PDF)
- Red Team Test Protocol (PDF)
- Premier Election Solutions (formerly Diebold) (PDF)
- Hart InterCivic (PDF)
- Sequoia Voting Systems (PDF)
UC Documentation Review Reports
- Premier Election Solutions, Inc. (formerly Diebold) (PDF)
- Hart InterCivic (PDF)
- Sequoia Voting Systems (PDF)
UC Accessibility Reports
Public Hearing
- Webcast of the July 30, 2007, Public Hearing - Transcript (PDF)
- Public Notice (PDF)
- Agenda (PDF)
- Written Public Comment and Testimony
Press Releases
- Video of Secretary of State Bowen's August 3, 2007, Announcement
- August 3, 2007 (PDF)
- July 27, 2007 (PDF)
- June 21, 2007 (PDF)
- May 10, 2007 (PDF)
- May 9, 2007 (PDF)
Testing Security Plans
- Secretary of State Testing Security Plan (PDF)
- Source Code Review Security Plan (PDF)
- Red Team Security Plan (PDF)
- Documentation Review Security Plan (PDF)
Additional Information
Following are a number of documents related to the review, including a "Frequently Asked Questions" document to help people understand how the review was designed.
- One-Page Summary - May 9, 2007 (PDF)
- Frequently Asked Questions - Revised August 15, 2007 (PDF)
- Frequently Asked Questions - Revised July 2, 2007 (PDF)
- Frequently Asked Questions - Revised June 13, 2007 (PDF)
- Frequently Asked Questions - Revised May 11, 2007 (PDF)
- Frequently Asked Questions - Revised May 9, 2007 (PDF)
- Contract Between Secretary of State and University of California Regents (PDF)
- Public Observation Guidelines (PDF)
Professional Qualifications of the Top-to-Bottom Review Teams
Documentation Review Team Members
The reviewers were responsible for analyzing voting system security, accessibility, usability, reliability, accuracy and protection of ballot secrecy based on relevant documentation. The reviewers had access to documents such as reports from Independent Testing Authorities (ITAs), reports and data from state certification testing, and documentation related to how the systems are designed to be used in an actual election. The reviewers were split into three teams of two or three members to review each voting system subject to the top-to-bottom review.
- David Wagner (PDF) Principal Investigator
- Aaron Burstein (PDF)
- Nathan Good (PDF)
- Joseph Hall (PDF)
- Candice Hoke (PDF)
- Dave Kettyle (PDF)
- Deirdre Mulligan (PDF)
- Laura Quilter (PDF)
- Tom Ryan (PDF)
Source Code Review Team Members
Source code is the computer language that effectively controls how electronic voting systems operate. The source code reviewers were split into three teams of six or seven members to review each voting system subject to the top-to-bottom review. Here is a more detailed description of source code review.
- David Wagner (PDF) Principal Investigator
- Matt Blaze (PDF)
- Joseph Calandrino (PDF)
- Arel Cordero (PDF)
- Sophie Engle (PDF)
- Ariel Feldman (PDF)
- J. Alex Halderman (PDF)
- Srinivas Inguva (PDF)
- Chris Karlof (PDF)
- Eric Rescorla (PDF)
- Naveen Sastry (PDF)
- Hovav Shacham (PDF)
- Micah Sherr (PDF)
- Till Stegers (PDF)
- Dan Wallach (PDF)
- Ka-Ping Yee (PDF)
- Harlan Yu (PDF)
- William Zeller (PDF)
Red Team Members
Red team members were responsible for testing the functions and performance of the voting systems and identifying security or accuracy vulnerabilities . The red team members were split into three teams to review each voting system subject to the top-to-bottom review. Here is an explanation of red team testing in general and the preliminary protocols that were used in the red team testing.
- David Wagner (PDF) Principal Investigator
- Robert Abbott (PDF)
- Elliot Proebstel (PDF)
- Sujeet Shenoi (PDF)
- Davide Balzarotti (PDF)
- Greg Banks (PDF)
- Marco Cova (PDF)
- Mark Davis (PDF)
- Viktoria Felmetsger (PDF)
- Richard Kemmerer (PDF)
- William Robertson (PDF)
- Jacob Stauffer (PDF)
- Fredrik Valeur (PDF)
- Giovanni Vigna (PDF)
Accessibility Team Members
The accessibility reviewers examined the voting systems subject to the top-to-bottom review to determine whether they were accessible to voters with disabilities and voters with alternative language needs.
Draft Criteria
The draft criteria for the top-to-bottom review of voting systems certified for use in California were released for public comment on March 22, 2007. The final criteria for the review can be found in the Contract Between Secretary of State and University of California Regents
- Press Release - March 22, 2007 (PDF)
- Draft Criteria Released for Public Comment - March 22, 2007 (PDF)
Public Comment
Counties
- California Association of Clerks and Election Officials (CACEO) (PDF)
- California State Association of Counties (CSAC) (PDF)
- Butte (PDF)
- Contra Costa (PDF)
- Kern (PDF)
- Los Angeles (PDF)
- Madera (PDF)
- Mariposa (PDF)
- Mendocino (PDF)
- San Joaquin (PDF)
- San Luis Obispo (PDF)
- San Mateo (PDF)
- Santa Clara (PDF)
- Santa Cruz (PDF)
- Shasta (PDF)
- Solano (PDF)
- Sonoma (PDF)
- Yolo (PDF)
- Yuba (PDF)
Voting System Vendors
- Avante (PDF)
- Premier Election Solutions (formerly Diebold) (PDF)
- Equalivote and VotePAD (PDF)
- ES&S (PDF)
- Hart InterCivic (PDF)
- Sequoia Voting Systems (PDF)
Organizations
- Black Box Voting (PDF)
- California Council of the Blind (PDF)
- California Foundation for Independent Living Centers (PDF)
- Coloradoans for Voting Integrity (PDF)
- Open Voting Consortium (PDF)
- Protection and Advocacy, Inc. (PDF)
- Secure Accurate Elections (PDF)
- Townsend and Associates (PDF)
- VerifiedVoting.org (PDF)
- Wellstone Democratic Renewal Club (PDF)
Individuals
- Comments from Individuals (PDF)
- All personal information has been removed from comments submitted by individuals.