Report on Claim of
ES&S InkaVote Plus System Failure

June 24, 2010

Introduction

On June 8, 2010, Mr. Brad Friedman published an account on his blog https://bradblog.com/?p=7884 of an alleged failure of the ES&S InkaVote Plus Voting System. Our analysis did not identify any systemic failures. Our analysis does, however, underscore the limitations and inadequacies of the Audio Ballot Booth system’s capacity to accommodate California’s complex Modified Closed Primary elections that, in this election, required nine different ballot combinations per precinct. Specifically, we found that the principle source of the failures identified by Mr. Friedman were the result of the incorrect ballot group/party code entered to initiate the specific ballot on the Audio Ballot Booth (ABB). This key-in error caused Mr. Friedman to be presented with a ballot group and party that was different from his assigned ballot group and party. The following account explains what likely happened while Mr. Friedman was attempting to vote using the ABB at his polling place, based on the narrative and pictures provided in his blog, examination of the Precinct Ballot Reader and ABB devices deployed to the polling location, interviews with poll Inspectors who worked at the designated polling location and a review of Election Day voting systems support logs.

Analysis of Mr. Friedman’s Voting Experience

Mr. Friedman went to his polling place (9001201A). This polling place was a Neighborhood Voting Center and, therefore, hosted an additional precinct (9000956A). Mr. Friedman requested to vote using the Audio Ballot Booth, and was the first voter to request use of the Audio Ballot Booth (ABB).

Failure #1: Mr. Friedman explains that his audio ballot started playing in “some kind of Asian language.” The Inspector then incorrectly provided Mr. Friedman with the instructions to get to an English ballot by pushing the arrow keys.

Findings: In actuality, the Inspector did not fully enter the correct information to bring up an audio ballot in English. Once one enters a ballot group and party code, referred to as the Audio Ballot Code, the operator must wait for the Precinct Ballot Reader (PBR) to prompt for the language of the voter’s choice. Instead, it is surmised that the Inspector became impatient and pressed the enter key a second time. By doing this, the language selection only briefly appears and the PBR immediately selects Vietnamese as the preferred language since it is the choice closest to the Enter button on the previous screen.

After the Inspector called into the RR/CC Help Desk, they were able to get the audio language changed to English.

Help Desk documentation of the Inspector’s call and post-election testing done by the RR/CC team of Steve Hodges, Joshua Bray, Jon Kato and Jeff Klein validated the key-in error.

Failure #2: Mr. Friedman reported that the audio ballot offered him candidates for the 31st Congressional District instead of his own. Mr. Friedman also noted that the candidates were offered in a different order than their listing in his sample ballot.

Findings: The appropriate Audio Ballot Code for Mr. Friedman’s precinct was 0611 as defined by the polling place ballot group and voter’s party code. The code that was erroneously entered into the PBR was 1201 (the last 4 digits of the precinct number). This number initiated a ballot from ballot group 120, instead of the appropriate ballot group number of 061. [Redacted for privacy]. Ballot group 120 is part of the 31st Congressional District, which Mr. Friedman was presented with, instead of the expected 28th Congressional District listed in Mr. Friedman’s sample ballot book. The different ballot group numbers also correspond to a different candidate order due to candidate rotation required by the California Election Code.

Testing done by the RR/CC team of Steve Hodges, Joshua Bray, Jon Kato and Jeff Klein validated this error.

Failure #3: Mr. Friedman then went to the other hosted precinct (9000956A) at the Neighborhood Voting Center and attempted to vote provisionally using the ABB device assigned to that precinct. They once again initiated an audio ballot only to have the party selected as Peace and Freedom party. [Redacted]

Findings: The ballot group for this polling place is also 061. Therefore, in order to get [the appropriate] ballot, the poll worker should have entered 0611 for the Audio Ballot Code as indicated on the ABB instruction page provided with the precinct election supplies. However, the poll worker again erroneously entered the last four digits of the precinct number (0956). The code 0956 instructed the ABB to provide the Peace and Freedom ballot for ballot group 095 (the last digit of “6” is the party code for the Peace and Freedom party). An ambiguity in the screen design and labels is likely responsible for the Inspectors’ making an assumption that the code to be entered should be based on the precinct number as opposed to the assigned Audio Ballot Code provided in the precinct voting supplies.

Testing done by the RR/CC team of Steve Hodges, Joshua Bray, Jon Kato and Jeff Klein validated this error.

Failure #4: After a second attempt, still using the incorrect Audio Ballot Code and receiving a Peace and Freedom ballot again, Mr. Friedman notes that the instruction page provided to poll workers was missing the Audio Ballot Codes.

Findings: The instruction page that contains Audio Ballot Codes, which Inspectors use (as shown in Mr. Friedman’s photo), was a sample page provided during Poll Worker Training Class. During training, all poll workers are provided this sample instruction page as an example. Audio Ballot Codes are intentionally left off this page to avoid any confusion since the actual codes are issued within each precinct’s supplies. Furthermore, during training class, it is explained to poll workers – on several occasions – that the instruction page they received during class is only a sample and one with their precinct’s actual codes will be included with the supplies. A review of the precinct materials returned to the Election Operations Center (EOC) from Mr. Friedman’s polling location confirmed that the ABB instruction page with the correct Audio Ballot Codes was in the materials for both precincts assigned to the Neighborhood Voting Center. Additionally, during poll worker training courses a hands-on session to initiate the Audio Ballot Booth is conducted, during which time poll workers have an opportunity to ask questions and observe.

It is our belief that the Inspector did not look in the precinct-issued supplies for the page containing the Audio Ballot Codes, but instead used the sample page provided during training since he was more familiar with that.

Failure 5: Mr. Friedman states that the PBR system does not have the ability to specify the party selection.

Findings: The audio ballot party selection is combined as part of the Audio Ballot Code. As noted in the “Operating the Audio Booth” instruction page, every party receives its own Audio Ballot Code. The Audio Ballot Code is comprised of the three digit ballot group number followed by a single digit party code to form a unique four-digit number for every single possible ballot group and party combination. As such, the party selection occurs as part of the audio ballot initiation process of a poll worker entering in the appropriate four-digit Audio Ballot Code. The header cards that activate the PBR and ABB and the Audio Ballot Code charts were provided for each precinct.

Failure 6: Mr. Friedman expressed his frustration at not being able to vote using an e-voting device; he gave up and voted manually. He states, there is no way to know the op-scanner will record his votes.

Findings: The polling place PBR system does not tally votes. All ballots are brought back to the Registrar-Recorder/County Clerk headquarters to be inspected, processed, and counted by our central tally Microcomputer Tally System. If the ABB had been properly initiated for an audio ballot, Mr. Friedman would have been able to vote per his sample ballot.

Testing

PBR and ABB devices, in both precincts, were isolated at the RR/CC’s Election Operations Center immediately following the election. These devices were used to perform the tests cited in our report and based on the alleged failures described by Mr. Freidman [sic] in his blog.

Conclusion

Both the analysis and testing confirmed that there was not a systemic failure of the InkaVote Plus System; nor in the pre-election loading of the data or in the software operation. The error was caused by a key-in error where the operator entered the wrong Audio Ballot Code (consisting of the ballot group number and party code) to specify the desired ballot style. A review of “Operating the Audio Booth” instruction sheet for the precinct reveals that the codes were correctly provided at the precinct.

This incident clearly points to design weaknesses and system limitations that exceed a reasonable level of usability and complexity of the Audio Ballot Booth during a modified closed Primary Election process and its associated variables (multiple party ballot styles and language requirement, for example). Additionally, it highlights a recognizable need for continued improvement of poll worker training and instructional handouts addressing the InkaVote Plus Voting System’s interface and the initiating of the ABB unit to assist voters, and where necessary, improved ballot adjudication procedures to ensure ballots that are created under such circumstances are detected and dealt with appropriately. While enhancements to training materials and improvements in instructional materials were implemented prior to this election, it is clear that there are system design and capacity issues that exacerbate the potential for operator error in the proper use of the ABB components of the InkaVote Plus Voting System.

Two design features of the systems are symptomatic of the failures identified. First, when entering the Audio Ballot Code, the software identifies the field as precinct number and precinct ID which are misleading to the poll workers. These fields should be labeled Audio Ballot Code. Second, it should be noted that the requirement to enter an Audio Ballot Code is only necessary for primary elections and when the number of ballot styles that are initialized on the PBR exceeds eight. With six qualified party ballot styles, the non-partisan ballot style, and two crossover ballot styles (NP Crossover-Democratic and NP Crossover-Republican), the number of ballot styles for this election totaled nine. With nine ballot styles used, the PBR required operators to specify the ballot style by entering an Audio Ballot Code consisting of the ballot group number plus the party code. When there are eight or fewer ballot styles, the software, as currently programmed, allows the operator to simply select a party name on the touchscreen to initiate an audio ballot. This is a more intuitive and user-friendly interface design than the complexity of entering Audio Ballot Codes that was required in this election.

A software change to allow the interface to support up to ten ballot styles before defaulting to the Audio Ballot Code entry screen was considered as a partial solution prior to this election; however current state and federal voting systems certification processes and timeframes do not allow for such changes without a full recertification of the system – a process that could not have been completed prior to this election cycle and would have required further contracting with a commercial voting systems vendor. Discussion with and consideration by the Secretary of State of changes to the software that would have enabled the validation of the Audio Ballot Code or otherwise eliminated the requirement of the operator (poll worker) to enter a code were deemed to require a full federal and state recertification of the system.

Two significant factors will impact future use of the ABB to comply with federal requirements for accessibility of voting as identified in the Help America Vote Act (HAVA) in Primary Elections. First, the passage of Proposition 14 and the establishment of a new “top two” model for primaries in California effectively eliminates the potential of exceeding eight party ballot styles that triggered the requirement of poll workers to enter an Audio Ballot Code as described above. Further, Los Angeles County is in the process of conducting a Voting Systems Assessment Project (VSAP) that will ultimately identify a process by which the county’s voting systems will be replaced and modernized. Accessibility and usability requirements for such a system are principle priorities for the project.