Specify minimum set of tables necessary for export for public Audit Center

[sfsinger19103]

Here's a minimum protocol which pretty much determines the minimum set of tables.

source: https://docs.google.com/document/d/1olo2W38kMi4mUH1S3UxRfFMlrszXHjLNK_dLuAMLT1w/edit#heading=h.ra35frr05gsz

Minimum protocol to check correctness of Colorado’s Risk-Limiting Tabulation Audit
If an independent entity wishes to check the correctness of the Colorado risk-limiting tabulation audit, it is sufficient for that entity to carry out the following steps. We believe these to be a minimum set of necessary items to verify that the audit was correctly conducted.

Note that this protocol does not address certain aspects of public verification of the election results, but only the correct performance of the Risk Limiting Audit from the ballot cards in custody of the Counties and the ballot manifest and CVR files exported from the voting system. In particular, chain of custody issues for the ballot cards before tabulation are not addressed by this protocol.

There is no break in the chain of ballot card custody between the tabulation of the results by the system under audit and the Audit Board review of ballot cards.
Observation: details of observation depend on chain of custody protocol.
Tabulating contests “selected for audit” on the CVRs used for the audit yield the announced tabulated results. Requires:
Data: CVR files
Data: List of contests selected by Secretary of State for audit
Data: Announced tabulated results for contests selected for audit
Calculation: tabulate results in contests selected for audit from CVR entries, compare to announced results. These should match.
The set of ballot cards listed in each ballot manifest exactly matches the set of ballot cards represented in the corresponding CVR file. Requires:
Data: The ballot manifest files
Data: The CVR files
Calculation: match the data lines in the CVR file one-to-one with the cards listed in the corresponding ballot manifest file
The list of ballot cards selected for audit has been determined after the finalization of the manifests and CVR files. Requires:
Data: Hashes of CVR and ballot manifest files (released to public before selection of random seed)
Data: full CVR and ballot manifest files (can be released later)
Calculation: calculate hashes of the full files and compare to the hashes released before the selection of the random seed. These must match.
Random sequence of ballot cards has been properly selected according to the specified random algorithm. Requires:
Observation: Random seed generated at public meeting
Data: Random sequence of ballot cards used for the audit. (This random sequence may include duplicates.)
Data: Ballot manifest file
Data: CVR file
Calculation: Use ballot manifest file to number ballot cards represented in the ballot manifest sequentially. Then calculate the pseudo-random sequence via prescribed algorithm based on the random seed. Together these determine the random sequence of ballot cards. This should match the sequence used in the RLA.
The list of ballots selected for audit matches the list actually reviewed by the Audit Board.
Data: CVR file
Calculation: match lines of the CVR file to the ballots listed in the ballot manifest.
Data: List of ballot cards assigned for review by Audit Board. This list can be created from the random sequence by removing duplicates.
Observation: physical assembly of ballot cards for Audit Board review
Audit Board interpretation matches content of paper ballot cards
Observation: Audit Board review of paper ballot cards
Audit Board intent is reflected in the computer record of Audit Board intent.
Observation: Audit Board review of ballot cards
Data: computer record of Audit Board review
Calculation: the records from the observations should match the computer record.
Risk limit achieved (or hand count ordered)
Data: status of each contest selected for audit -- in particular, which contests if any has the Secretary of State selected for hand count?
Data: the “risk limit” for each contest selected for audit.
Data: the “error inflation factor” used to calculate whether the risk limit has been achieved
Data: the “diluted margin” for each contest under audit, i.e., the vote difference between the lowest winning total and the highest losing total, divided by the total number of ballot cards in the county. This can be calculated from the CVR file.
Data: for each contest under audit, the number of each type of discrepancy found in that contest by comparing Audit Board interpretations to the original CVR file. These can be calculated by comparing the entries in the CVR file to the corresponding entries in the record of Audit Board interpretations. The types of discrepancies are:
Two-vote overstatements (Audit Board finds a vote for a loser where original CVR showed a vote for the winner)
One-vote overstatements
One-vote understatements
Two-vote understatements
Calculation: Using the formula for the stopping size given on Stark’s Audit Tools web page, see if the size of the sample checked so far exceeds the stopping sample size. With the following notations
a risk limit
g error inflation factor
m diluted margin
o2 number of two-vote overstatements
o1 number of one-vote overstatements
u1 number of two-vote understatements
u1 number of two-vote understatements

the formula for the stopping sample size is:

stopping sample size = -2g(log(a) + o1log(1-1/(2g)) + o2log(1 - 1/g) + u1log(1+1/(2g)) + u2log(1+1/g)) / m)

[sfsinger19103]

And here are the data specs pulled out of the protocol:

• Data: CVR files
• Data: List of contests selected by Secretary of State for audit
• Data: Announced tabulated results for contests selected for audit
• Data: The ballot manifest files
• Data: The CVR files
• Data: Hashes of CVR and ballot manifest files (released to public before selection of random seed)
• Data: full CVR and ballot manifest files (can be released later)
• Data: Random sequence of ballot cards used for the audit. (This random sequence may include duplicates.)
• Data: Ballot manifest file
• Data: CVR file
• Data: CVR file
• Data: List of ballot cards assigned for review by Audit Board. This list can be created from the random sequence by removing duplicates.
• Data: computer record of Audit Board review
• Data: status of each contest selected for audit -- in particular, which contests if any has the Secretary of State selected for hand count?
• Data: the “risk limit” for each contest selected for audit.
• Data: the “error inflation factor” used to calculate whether the risk limit has been achieved
• Data: the “diluted margin” for each contest under audit, i.e., the vote difference between the lowest winning total and the highest losing total, divided by the total number of ballot cards in the county. This can be calculated from the CVR file.
• Data: for each contest under audit, the number of each type of discrepancy found in that contest by comparing Audit Board interpretations to the original CVR file. These can be calculated by comparing the entries in the CVR file to the corresponding entries in the record of Audit Board interpretations.

[nealmcb]

Note that the "source" draft document referenced above ("Public oversight protocol" on Google Drive) includes suggestions, updates and discussion, and is easier to read than the version here.

See also #601, which defines a similar set of requrements and includes some related queries.