A risk-limiting audit is a method of ensuring that election results match voter selections reflected on paper ballots.

For California counties that choose to conduct a risk-limiting audit, using statistical sampling techniques, elections officials review a sample of ballots cast in an election to confirm that the reported results tabulated by the voting system are accurate. Additional ballots may be reviewed as necessary until the results of the election are confirmed to a certain degree of confidence.

In California, when a risk-limiting audit is used, it must confirm that the election results reported by the voting system are 95% likely to be accurate. Unlike the post-election 1% manual tally traditionally used to confirm election results in California, under a risk-limiting audit every ballot cast in the election—regardless of which precinct it was cast in—has an equal chance of being audited.

The following flowchart shows how risk-limiting audits are conducted:

risk limiting audit flow chart

Source: https://www.verifiedvoting.org/

California Pilot Program

In California, a county may either conduct a risk-limiting audit or a 1% manual tally. (See, Elec. Code, §§ 15365-15367.) The risk-limiting audit pilot program will sunset on January 1, 2023.

The Secretary of State has promulgated regulations that implement and administer the risk-limiting audit pilot program. Those regulations can be found at: https://www.sos.ca.gov/administration/regulations/.

Risk-Limiting Audit Process in California

The process of conducting risk-limiting audits in California is summarized below. This process is available for public observation.

  1. Public Notice of the Audit
    Any county that chooses to participate in the pilot program must provide at least five days public notice. This webpage will list each county that has chosen to participate.
  2. Audit Board Selection
    The county elections official will appoint people to review ballot cards and identify voter choices. No fewer than three people will review each ballot, but the same people do not need to review every ballot. Audit boards may be comprised of volunteers from the public and elections office staff. All audit board members must complete and sign a declaration of intent to faithfully discharge audit board duties.
  3. Generation of the Random Seed
    Members of the public will be invited to take turns rolling 10-sided dice to generate a random number (or seed). The audit software will use this seed to randomly identify ballots to audit.
  4. Entry of Election Data into the Audit Software
    The county elections official will enter information about the election into the audit software that allows the audit software to randomly identify ballots to audit.
  5. Identification of Ballots to Audit
    The audit software will determine the number of ballots needed to audit in order to confirm the result of the election to a 95% certainty. The audit software will randomly generate a list of ballots to review.
  6. Ballot Examination
    Audit board members will review each ballot selected by the audit software and determine voter choices on all contests being audited. The county elections official will resolve any disagreements among audit board members.
  7. Entry of Voter Choice into the Audit Software
    The county elections official will enter determinations of voter markings made by the audit board into the audit software. The audit software will use this information to determine whether the expected result of the election is confirmed by those voter selections.
  8. Statistical Confirmation of Election Results
    The audit will continue until the audit software can confirm the results of the election based on the voter choice determinations made by the audit board on the ballots they review. If the results represented on the ballots reviewed do not agree with the election results, the audit software will generate a list of additional ballots to review. This process will continue until either the results of the election are confirmed, or all ballots cast in the election are reviewed.
  9. Reporting of Audit Results
    The county elections official will report the results of the risk-limiting audit in the certification of the official canvass of the vote, which is due to the Secretary of State within 30 days of the election. This report will include information about the ballots reviewed and anything unusual that occurred in the audit. The results of all audits will be posted on this webpage.

Risk-Limiting Audit Software Tool

All counties participating in the pilot program shall use an RLA software tool provided by the Secretary of State to conduct the audit. The Secretary of State contracted with VotingWorks for the 2020 and 2021 elections to provide a web-based tool based on open-source and freely available code. This tool is called Arlo.

More information about Arlo can be found on the following web page: https://voting.works/risk-limiting-audits/.

Audit Results

All counties participating in the risk-limiting audit pilot program must report the results of the audit in the certification of the official canvass of the vote. Information about the official canvass of the vote can be found at: https://www.sos.ca.gov/elections/official-canvass/. 

Audit results for the California Presidential General Election - November 3, 2020 

Audit results for the California Gubernatorial Recall Election - September 14, 2021

Audit results for the Shasta County Supervisor District 2 Recall Election – February 1, 2022

Audit results for the California Statewide Primary Election – June 7, 2022

Audit results for the California Statewide General Election – November 8, 2022

This webpage will be updated with certifications of the official canvass of the vote for each county that participates in the pilot program after those counties submit their certifications to the Secretary of State.

Additional Information about Risk-Limiting Audits

The difference between a 1% manual tally and a risk-limiting audit:

  • 1% Manual Tally: Elections officials conduct a public manual tally of 1% of all ballots tabulated on a voting system during an election, including vote-by-mail ballots. This is a public process of manually tallying votes in 1% of the precincts, selected at random by the elections official, and in one precinct for each race not included in the randomly selected precincts. This procedure is conducted during the official canvass to verify the accuracy of the automated count. In a regular election year, counties hand count tens of thousands of ballots as part of the 1% manual tally. This process does not provide statistical evidence that the machine tally found the true winner for each contest on the ballot. This process also does not describe what should be done if the results of the manual tally do not agree with the election results. (See, Elec. Code, § 15360.)
  • Risk-Limiting Audit: Elections officials manually tally randomly selected ballots, stopping as soon as it is implausible that a full recount would show a different result than the ballots reviewed. As long as it is statistically plausible that a full recount would overturn the result, the risk-limiting audit continues to examine more ballots. Risk-limiting audits determine precisely how much hand-counting is necessary to confirm election results to a given level of confidence. The closer the contest, the more ballots that must be examined to have strong evidence – because fewer errors can change the outcome. The higher the desired confidence (e.g., 99% versus 90%), the more ballots that must be examined – because higher confidence requires more evidence. State law requires a 5% risk limit, or 95% confidence in the result of the election reported by the voting system. (See, Elec. Code, § 15367).

Methods for ensuring ballot security and chain-of-custody

Ballot security and chain-of-custody for ballots is essential for ensuring that the ballots are secure and have not been tampered with. The following methods achieve these goals:

  • Ballot containers (bags, boxes, tubs, etc.) should be properly identified, signed, and sealed by at least two elections officials.
  • A chain-of-custody log should accompany every container, and seal numbers should be confirmed by elections officials at the ballot storage facility.
  • Elements of the chain-of-custody log may include the precinct ID, number of ballots in each container, and the tabulator ID. The same principles apply to voting equipment chain-of-custody. 

Additional References