• Safe and secure

  • Quick and easy

  • web-based solution

  • 24/7 Customer Service

Rate form

4.1 Statisfied

370 votes

To Fill In Fillabe Oregon Request To Energize An Electrical Form , Follow the Steps Below:

Create your Fillabe Oregon Request To Energize An Electrical Form online is easy and straightforward by using CocoSign . You can simply get the form here and then write down the details in the fillable fields. Follow the instructions given below to complete the document.

Fill out the customizable sections

Customize the form using our tool

Fax the completed form

  1. Look into the right document that you need.
  2. Press the "Get Form" icon to get your file.
  3. Check the whole form to know what you need to key in.
  4. Enter the information in the free-to-edit parts.
  5. Double check the important information to make sure they are correct.
  6. Click on the Sign Tool to design your own online signature.
  7. Drag your signature at the end of the form and press the "Done" button.
  8. Now your form is ready to print, download, and share.
  9. If you have any doubts regarding this, don't hesitate to contact our support team.

With the help of CocoSign's eSignature solution , you are able to get your document edited, signed, and downloaded right away. All you have to do is to follow the above process.

Thousands of companies love CocoSign

Create this form in 5 minutes or less
Fill & Sign the Form

Hand-in-Hand Teaching Guide to key in Fillabe Oregon Request To Energize An Electrical Form

youtube video

Fillabe Oregon Request To Energize An Electrical Form Demand Assistance

.. >> We're going to .start in about two minutes. In .about two minutes, we're going .to start..>> Good morning, everyone. I .love the fact that people get .quiet when I get up on stage. .Good morning. I'm Tom /HEUBG. .I'm chairman of the United .States election commission..I should have asked the.folks if we're streaming before .I get going. So good morning..I'm Tom Hicks, chairman of the .United States election .commission, or EAC. And I want .to take one minute to pledge .allegiance to the flag, which is.right behind me here. .I pledge allegiance to the flag .of the United States of America,.and to the republic for which it.stands, .one nation, under God, .indivisible, with Liberty and .justice for all. . So with that, I want .to ask everyone to silence their.phones so that .we don't have any mishaps during.the summit. .We are honored to be hosting .this Election Data Summit, in .partnership with the .Pennsylvania Department of .State..The EAC held its first data .summit in .2015, and we're excited to carry.the momentum toward this event .today. Thank you all who have .joined us here today in the .audience and those of you who .are watching this live streaming.from home or in the office..Collecting, analyzing, and .sharing election data is a big .part of what we do here at the .EAC. .In service of our clearinghouse .function and all the mandates .under the Help America Vote act,.which administer the election .administration and voting .survey, or EAVS, after every .federal election..The EAVS survey collects state .by state data under jurisdiction.of federal elections in all 50 .states, the District of Columbia./KWRA, and four U.S. .territories..The EAVS contains the most .comprehensive nationwide data .about how elections are run in .the United States, .data on voter registration, .voting by military and overseas .citizens, absentee, and .provisional ballots, polling .operations, election day .activities, and voting .equipment. . The EAVS provides .policy makers and the public .with key information about how .our democracy functions..We at the EAC take our .responsibility as stewards of .critical national election data .seriously, and we are busy .laying the groundwork for the .2018 survey. Building on some .important changes for 2016, .we're working to improve the .survey in three ways. First, .we're working to make the survey.easier for election officials to.complete. Second, we're taking .steps to strengthen the data .quality and completeness. And, .third, we're working to Foster .greater use of the data by .experts and election officials .alike..We know that EAVS data barely .scratches the surface of the .wide world of election data .that's out there. State and .local election officials across .the country use data every day .and are constantly innovating to.improve election -- page turn --.processes. . From voter .registration processes to .post-election review .exercises, data driven .approaches are helping election .officials identify trends, .anticipate changes, voters need,.predict the impact of proposed .policies, and determine how to .invest resources. As election .officials prepare for the .upcoming midterm elections, we .are thrilled to be partnering .with our friends at the .Pennsylvania Department of State.to offer this platform for .election officials and election .data experts to share their .experiences and lift up .innovation, data driven .practices from across the .country. . With that, I'm going .to talk about a few housekeeping.things that are going on. .If you are a social media user, .we encourage you to post about .this summit using the .#electiondata18..Again, that's #electiondata18.. There are a few .opportunities throughout the day.for our audience to ask .questions of our panelists. If .you would like to ask questions,.we politely ask you to do one of.three things. This is about .data, three things. Remember .this. One, speak into the .microphone so that folks who are.listening from home can actually.hear the question..Two, identify yourself, name and.affiliation..And, three, this is most .important, actually ask a .question. . If you need assistance.throughout the day, please find .a staff member from the EAC or .the Pennsylvania Department of .State, because we're all happy .to help and assist you. . Thank you again for .coming, and thank you for the .Community College of .Philadelphia for hosting us here.today. Let me give a special .note of thanks for the election .officials who are here. And if .you're an election official and .wanting to stand for a quick .acknowledgement, please do so. .Don't be shy. Don't be shy. .[ Applause ].. Most of these folks .are here .for the NAS and NASED .conference, but we're very .encouraged they all came for our.summit today. .Finally, let me give a sincere .appreciation for acting .secretary Robert Torres and his .team at state for inviting us .here today to the commonwealth .department for this important .initiative. At this time, it's .my pleasure to introduce .secretary Torres, who will be .delivering opening remarks. .Thank you once again to the .audience in your commitment to .secure accurate and improving .elections. Thank you. . >> Good morning. .Chairman Hicks, it's good to see.you. Thank you for being here..On behalf of the Department of .State, it's my privilege to .welcome you to the .city of Philadelphia, the .Community .College of Philadelphia for .today's Election Assistance .Commission..Our happy to partner with the .election commission on this data.summit co-hosted by the state .and the EAC. I would like to .extend my gratitude and a .special thanks to our deputy .policy director Jessica Myers .for all of her hard work and .collaboration with our .colleagues at the EAC to help .organize and bring you this .relevant, educational, and .timely summit. . Of course, I also want.to thank the EAC team for their .hard work and support in putting.today's summit together..And by the way, you can't beat .the price. So there should be .no complaints, at least on that .front. .So let me also extend my thanks .to all the moderators and the .panelists for your time and .effort in participating today. .And finally, thank you all for .your interest and attendance in .what I trust will be a .worthwhile investment of your .time. . Pennsylvania takes the.administration of our elections .very seriously. We at the .department work in partnership .with counties to ensure smooth, .fair, and secure elections. .However, I know that many, if .not all of us, want to find more.efficient ways of administering .elections and managing our .limited resources. This is why .we have been working to .modernize how we manage data and.information in Pennsylvania. .With regard to voter .registration, we .implemented online voter .registration .nearly three years ago and .passed the 1 millionth user mark.last February. We have also .developed web APIs or interfaces.to further continue and leverage.the progress that we have .achieved with online voter .registration. .However, the voter registration .database that we use needs .to be replaced because it is .costly and doesn't support the .kind of information reporting .and analytics that a modern .system could provide. .Therefore, we are in the process.of developing business .requirements with strong .considerations on how we want .the data and the information .derived from that data to work .for us and the counties so that .we can maximize the .benefits of a new system. . On election day, we .work with the counties and other.state agencies, such as the .office of administration, .Pennsylvania Emergency .Management .Agency, office of Homeland .Security, and others to monitor .events throughout Pennsylvania .that could impact voting..The surveillance and use of data.in .the communications channels we .have established, help to .maintain situational awareness .and help us respond quickly to .problems. . Pennsylvania is also .pursuing the transition to new .voting system within the next .two years. We have directed .that all new voting systems .purchased in the commonwealth .must have voter verifiable paper.ballot or paper records of votes.cast by the voter that will .enable us to audit the systems .and conduct post-election audits.that will help further validate .voting results and maintain .confidence in the integrity of .the electoral process. .ADA and the effective use of .data to help manage all aspects .of the electoral process is .extremely important. However, .data alone is not enough. The .quality of the data you use .always has to be considered. .The skill sets of the staff and .analysts who use the data that .you will rely upon for .information has to be .considered. How you formulate .research questions has to be .considered because, if you ask .the wrong questions, guess what?.The data will be useless to you.. Voter confidence in .our elections can be impacted by.their experience at the polls .and their observations on how .elections are handled..As election officials, .advocates, and stakeholders, it .is incumbent on us to .collect and use data effectively.to help .improve our properties, enhance .resource allocations, increase .accuracy, and support .auditability. All of this .should be done to ensure voters .and ourselves that the integrity.of procedures being used and .ultimately our elections can be .trusted. . Let me just say this. .If you don't remember anything .else I say this morning, please .remember these words throughout .the day. If you cannot measure,.you cannot manage. I'll repeat..If you cannot measure, you .cannot manage.. Thank you for being .here. I wish you a productive .and enjoyable day. For those of.you who are staying for .the NAS or the NASED conference,.enjoy your stay and have a great.conference experience. Thank .you. .[APPLAUSE].. . >> Good morning. And .welcome to our first panel. My .name is Christie McCormick, and .I am the vice chair of the U.S. .Election Assistance Commission..Since the EAC was reconstituted .with commissioners in 2015, .we've taken a .number of steps to strengthen .our data collection efforts at .the EAC and to .encourage data driven practices .among election officials..In addition to our work to .improve the election .administration and voting .survey, we organized the EAC's .first .ever Election Data Summit in .2015. .And since I've been in this .business, I've realized that .election .folks are the most OCD people I .have .ever met, and this just feeds .that disorder..Some of the speakers here today .were .with us at the original summit, .Amber .McReynolds, Jennifer Morrell, .and I'm pleased there are many .new voices on the stage and in .the room today. We're proud to .be furthering this effort with .the Election Summit. Thank you .for the Pennsylvania Department .of State for partnering with us .on this important event. . The panels we have .arranged for you today follow .the phases of the election .cycle. We begin the day focused.on preregistration activities, .such as voter registration, and .we'll finish .with post -election matters, .including audits and after .action review processes. . This panel that we're .on right now will focus on the .topic of voter registration. .Voter registration data is the .backbone of election data. Not .only does this registration data.serve as the list of eligible .voters .used in polling put stations on .election day, but it is also .essential to vote by mail and .ballot creation..This data also drives key .election administration resource.key allocation decisions, such .as assigning poll workers and .voting equipment to polling .places. Statewide voter .registration databases are also .one of the main tools that .election officials have for .sharing election data within and.among states. . Our panelists here .will explore how voter .registration data is being used .across the country. This will .include a discussion of recent .efforts to modernize voter .registration systems and .processes as well as initiatives.to increase the .sharing of registration data .between states to support .improved list maintenance and .outreach to unregistered voters..Our esteemed panelists currently.serve -- or previously served as.election officials in Colorado, .Florida, Oregon, Pennsylvania, .and Virginia. Each of these .expert panelists brings a unique.perspective of the conversation .about data driven approaches and.improving voter registration .processes. . So I would like to .introduce our panelists before .we get started..On my far right is Mike Moser, .the .deputy commissioner of elections.in the Pennsylvania Department .of State. Welcome, Mike. He .joined the department in 2015 .and oversees and coordinates .elections and voter registration.initiatives and .federal and state elections with.the 67 county board of election .in Pennsylvania. . Next to him, to my .right again, is Dr. Judd Choate..Judd is the state election .director for Colorado and the .immediate past president of the .national association of state .election directors. Judd has a .J.D. from the University of .Colorado school of law and both .a Ph.D. and M.A. in political .science from Purdue University. .He was formerly a professor of .political science at the .University of Nebraska. In a .previous life, Judd was a scout .for the Kansas City Royals. And.as I found out this morning, he .also used to deejay for an adult.contemporary music station..So we had a nice conversation .about Neil Diamond. .Ericka Haas on my left is the .systems engineer and technical .liaison for the electronic .information registration center,.also known as ERIC. She manages.the data exchange of voter .registration and vehicle .licensing data for 24 states..Before joining ERIC, Ericka, .appropriately named, worked on .the Oregon state registration .system with the Oregon secretary.of state. .And then on my far left is Don .Palmer. He's a policy adviser .with the U.S. Election .Assistance Commission. He's a .fellow with the bipartisan .democracy project, focused on .the recommendations of the .presidential commission on .election administration of the .PCEA. Don previously served as .secretary of Virginia's board of.elections, and as director of .elections in Florida. He's also.served as a trial attorney with .the voting section in the .department of justice's civil .rights division, and we actually.started on the very same day at .the DOJ. So I've known Don for .quite some time. And as of .yesterday afternoon, he was also.nominated by the president to be.our next commissioner at the .EAC. So congratulations, Don. .We'll see how that goes. . [APPLAUSE]. .The way this is going to work is.each of our panelists will give .prepared remarks, and then we'll.shift into a question and answer.period. I have some questions .for them, and .then we will open it up to the .audience. So, Mike, you're up .first. Thank you.. >> Good morning, .everybody. It's great to see .you..Thank you so much for the .opportunity.. Fantastic. Thank you. . So I'm here to talk .about kind of the recent .modernization efforts for voter .registration in the Commonwealth.of Pennsylvania and kind of how .we're trying to synchronize what.we call the .service channel for voter ./REPBLG /STRAEUGS. And .reviewing the data and the .design of the applications we're.using to collect the data to .inform decisions and next steps .of where we want to take them. . So I'll call a journey.that we've been on to modernize .here in the Commonwealth, it's .been a great experience, and .what we're trying to do .is take a look at the paper, .traditional paper method of .registration versus online voter.registration and really trying .to hook into some other areas in.addition to the work that we've ./PW-P .doing with PennDot and the motor.voter transactions to actually .engage more .with registration drives and .third party entities. .We can kind of summarize it .into three broad areas we're .trying to focus on when trying .to modernize here in the .Commonwealth, and that's .usability, integrity, and .improvement of the forms. And .just the cycle in general of how.we're making changes. We're .really trying to focus on .usability, where we're focusing .on plain language, we're making .sure the applicant is .understanding the context of the.information that we're trying to.convey so they can submit their .information on the forms, and .also engage into user testing .with the entity so we can make .adjustments on the form to .collect the necessary .information to .ensure proper /REPBLG /STRAEUGS..That not only helps out with .them getting registered to vote,.but it also helps out our 67 .boards of election so they can .process the registration .adequately, and the voter .ultimately gets the registration.card, and they can go vote. . We're also focusing on.the integrity of the process. I.mean that loosely. That's where.we are looking at the different .types of data that's coming in .or /*. Are the data points .complete? Are they accurate? .Are they consistent? And then .we just kind of apply like the .lien process overall. I know .that's becoming more of a common.terminology among election .directors and just government .services in general. It's a big.initiative here in the .Commonwealth of Pennsylvania to .engage .in a lien or sig sigma, for .those who are familiar with it. .So we kind of look at the .overall process to ensure all .the information is there for the.applicant. .And then just continuous .improvement overall. We have an.agile environment here, and .that is a 30-day sprint cycle .for development here for our .online forms, .but we also apply it through .lien to make sure we're .continuously reviewing the .changes we're making. .So I just want to give a little .background on kind of the major .voter registration changes that .have occurred in the .Commonwealth. In 2003, we've .had an electronic .exchange through our PennDOT .transportation partner with .Moater Motor Voter. We've had .signatures coming over, and .we've made changes, which I'll .go over momentarily. In 2005, .all counties completed the .transition to our current voter .registration database, which is .what acting secretary Torres was.mentioning we're going through a.process to modernize the .database as a whole. .And then the bulk of what we've .been doing, the modernization .efforts came, started happening .in 2015, where we redesigned our.paper application with some .usability testing..We launched online voter .registration in the state, and .as the secretary noted, we went .over 1 million applicants, which.was a great experience for us. . In 2016 we simplified .the language for Motor Voter. .We incorporated online signature.upload feature into our online .voter registration application, .and I'll talk more about that .soon. And also implemented an .application programming .interface for third parties to .plug into for voter .registration. .2017, we built upon that to .engage with our state NVRA .agencies and adopted our first .agency into web API for online .voter registration. .Then in 2018, we launched a .third party feature for online .registration drives to help .there with our counties and the .applicants because, as election .officials are aware, usually .registration drive goes out on .paper and not all the necessary .information or the quality isn't.always the same and consistent .as would a normal signup. So we.want to kind of help out there .with the data we were seeing and.built an online feature to help .make it more consistent for the .applicant. .So this is just a quick example..I think on the left for you is .the old paper form that we had..The new form is.on the right for usability .testing. It's a lot more .consistent, and we've seen a .much better back end with data .there with the changes, and then.we also have examples of this is.what our current online voter .registration state looks like, .again with testing and review .and looking at the different .data points. We'll be pushing .out a new redesign in the future.and really cutting out a lot of .the extra space. .The idea I'm trying to drive is .that with data collection, it's .not just looking at the back end.information .that's coming in, but you can .also push certainly types of .collection and get better .quality data through initiatives.of the designs of forms you're .putting out there to help with .the back end analyses that a lot.of the experts will be doing and.to build a solid foundation for .the future. Design is certainly.an aspect that's been very .beneficial for us in .Pennsylvania. . So something that we .noticed right away and we were .kind of aware with online voter .registration when it was .launched, when somebody goes to .sign up online, not necessarily .versus a .paper form, if they didn't match.with PennDOT, they didn't have .an opportunity to bring their .signature over. Here in The .Commonwealth, if you match .with your PennDOT information, .your signature will .automatically come over on the .record. When OVR was first .launched here, that wasn't a .feature. So we were tracking .the information that was coming .through, and what I have .here since 2015, '16, and '17, I.have the total applications that.were coming through for online .voter registration..And then the number of missing .signature requests that would .come through. . Because what you could.do is, when you went through the.application .and you didn't match with .PennDOT at .first, you could request a mail .form come out, and basically the.applicant would sign it with key.information about .them and mail it back to their .county registration office. So .that was a process we were .really starting to look at where.people weren't always returning .the form. A lot of applications.were sitting out there, and we .really wanted to make sure the .applicant got registered to vote.but also the counties had what .they need to process the .application..So we kind of looked at it and .incorporated a signature upload .feature. . This is just a brief .screenshot. Apologize for the .blurriness a little bit. .Essentially, they will be .prompted at some point to upload.a signature, going through the .process and validations, and .then you can upload a signature .there..It's a little better than a .doctor's signature, but that's .the gist of it..You take a picture, and you can .submit it, and it comes into the.application for the county .registration office to see it .when they're reviewing the .information when they're in the .database. . Since we launched that.feature, we have seen the total .number of signature requests .come down. Initially in the .first year, it was about 20 .percent of total of online .applications. Since the feature.came out, we're now sitting at .about 12 or 13 percent. So it's.been a very big boost here by .changing the design of the form .and prompting the applicant .along the way, .we've been able to really get .better information and .registrations out of the online .process.. Overall with what .we're seeing with the online .application, we have more .approval ratings. Blue is the .paper method. Orange is the .online method. We have less .declinations coming out of the .system because we're putting .more validations up front for .the applicant as they're coming .along through the process, their.data is being checked with .PennDOT up front, and they have .the option with signature .upload. So we're getting more .complete applications coming .through, which has been very .helpful. .Now, this slide is really .where I think has been a big .boost to us .with looking into web .application interfaces and .integrating with third party .agencies for voter registration..By pushing more and modernizing .towards an online platform and .making sure that not only do we .have a common language across .all of our forms and methods of .delivery, but we're working .with third party entities to .make sure they're having similar.language on the form so it's a .consistent experience. And what.we've noticed with the design .and the collection is that the .online platform and third party .entities have .been really benefitting us in .what we're getting into the .system, which is very helpful .for any maintenance practices or.any matching we need to do to .review the information or to .look up information on the .voter. .We are overall seeing pretty .much an almost 90 percent of .applications coming in through .online .voter registration have a .driver's license..We're seeing almost 35 percent .that .always have at least an SSN on .there, last four of the social .security..And then online platforms .overall are really driving the .information of the individual .for contact, like an e-mail or a.phone number. And a lot of the .third party entities are really .helping out there, especially .on the registration drive, .because it's so core to the .business to be able to reach out.to the individual. .So I guess the point I'm trying .to drive there is that by .looking .into third party sources in a .secure manner and a consistent .manner and making sure that .they're going through .and testing it properly and .testing the .language of the forms, you can .really .broaden the information and have.more complete information coming.into the database that's not .only helpful to the applicant to.get registered, for the counties.to process, but it helps with a .lot of other initiatives just by.focusing on user testing and .designing the form in such a way.that they can better understand .the context. . There are some other .realized benefits we're seeing .with driving more .towards a modern platform with .online registration. We're .seeing that the application .process is becoming more .elastic, and I'll talk to that .in a second. Overall, .applications are more traceable .for the end user if they're .provided with an application I..D. where they can kind of go in .and type in some information .about themselves and look up the.status of their application..And then there's better .opportunities for us to .understand some behaviors of the.applicants and when they're .using the applications because .with a traditional paper method .you don't always have the same .benefit as it depends on when .it's arriving in the office, if .it's mailed in, things like .that, versus online you're .getting that data right away, .the time stamps and .things, so you can better .understand the behaviors of your.applicants. . So this is kind of .what I was .getting at with the elastic .process is that this is .preliminary. We surveyed all .the counties..And on average for every ten .paper applications that they are.processing, it takes them about .18 minutes in the Commonwealth .of Pennsylvania. And then .likewise, for an online .application, for every ten .applications they're processing,.it takes about 6 minutes..So we have seen the process come.down almost two-thirds, which .has been pretty impressive here .in the state, and I really have .to give credit to the county .board of elections for being .early adopters and being helpful.throughout the process and .embracing it. They have done a .tremendous job day in and day .out executing everything. So .thank you to them. .But to drive the point even .further, this is kind of the .loads of applications that we .cecuming in, where the green .line is the paper applications .from 2008, and everyone knows .that was a very heavy .registration year across the .nation. And then in 2012, which.is the blue .line and 2016 is the black line,.we had relatively similar .paperer applications coming in..And then the yellow or gold is .online voter registration itself.for 2016..And then the pink line is online.voter .registration with paper in 2016,.and it .is tremendous ly out soared the .2008 number. And by having a .more modern platform and .focusing on the design of the .form for both the front user and.the end user, the process was .able to become more elastic, .where a lot of the counties felt.they were in a good position in .2016 at registering voters right.around the deadlines because .they saved a lot of processing .time, and they had the .information they needed to .process those applications. . Both the commissioner .and I did a lot of phone calls .to talk and make sure everyone's.all right, and it just saved a .lot of time with processing .those applications. It would .have been a completely different.story if we would have had just .more of a traditional method .here in 2016. . And this is another .chart where we just kind of look.at the .behavior of the applications .coming in and the applicants, .whereby hour, since the .beginning of January 1st of .2017, we kind of form a behavior.of the submissions that are .coming in to help us inform .maintenance windows or impacts .the applicants that we have to .change anything. So it's just .another way that we're .kind of utilizing the data to .find some patterns so we .minimize impacts to the end .users. .And then also, we kind of see it.by the day of the week as well. .Monday is the most productive .day of the week, and we're .actually seeing that trend on .the applications. Everyone must.be thinking about it then..And then around Fridays and .Saturdays, things dip down to a .more slow period..So we actually not only look at .our resources, but we plan our .department development windows .around this type of information..We usually do releases around .the Thursday now because we see .things like this. . So I just kind of want.to close out because I know we .have a short .allotted time here to just kind .of say -- and I know it's .glossing over some items here. .Like find me on the side. I'd .be happy to give more .information .or ask some questions during the.Q&A period because there's a lot.of great things that modernizing.a platform can do by engaging .your end users. And by focusing.on the design of the form, you .can really improve the types of .information coming into the .voter registration database and .also inform some needed changes .that need to happen to your .applications by looking for .certain markers like application.statuses as they're coming in. . So by using data, it .really helps drive initiatives .but also at the .experience for the end user. So.I just want to say thank you so .much..Look forward to chatting with .you. .>> Thanks, Mike. I appreciate .it. Good job. .[APPLAUSE]. . >> You can see what .I'm .talking about, election geeks .being OCD. Next geek up is Judd.Choate. Judd, take it away.. >> Thanks. I'm Judd .Choate from Colorado..Let's see if we --.I have a couple of different .topics. What we collect and why.we collect them in the realm of .voter registration. .The first one, we try to .anticipate peaks in voter .registration. This kind of dove.tails a lot off of what Mike was.just saying, but we have tracked.a couple of things that I think .you might find interesting. We .communicate this with our .counties. . So the first one is .the Facebook effect, which I .have two examples of..In 2016 you can see our online .voter registrations for about .three or four .weeks or so, and on one .particular day we had over .20,000 updates or new .registrations. That was the .Facebook voter registration .advertisement. This is when .they sort of posted on .everybody's Facebook page that .they .should really go update their .voter registration or register .new if they hadn't. Google also.did a similar advertisement or .notice on their page, and then .there was national Voter .Registration Day, which followed.the Google day. These were all .over the course of five days in .September of 2016. .You can see that there was an .incredible jump that one day. .The only bigger day we had that .year was on election day 2016. .Colorado is a same day .registration state..So we get a lot of activity on .election day. .Then we also have the Facebook .effect during this 2018 primary..So our 2018 primary was on June .26, which is on the far right of.your screen. You can see that .more people registered to vote .or updated their registration on.the Facebook notice day than did.on election day. We also had a .big bump on the first day of .early voting..So that was the day that.that the VSPCs, voter polling .places, .opened on the 15th..So priming them there's a really.important day coming can get .them primed and understand they .need to bring in additional work.force or at least prepare .themselves for that particular .day..So that's one way we use data .for voter reg. .We also try to use it to .influence legislation which .you'll see that's varying levels.of success. .This is the 2016 presidential .election use of our polling .places to do updates .or new registrations during the .early voting period prior to the.election. You can see that a .lot of our activity occurs on .election day or the day prior .to election day, so Monday or .Tuesday of the election period, .and not a lot of activity .occurred prior to that. And I .break that down by the kinds of .activity that was occurring .during that time.. So updating existing .voter registrations is the .yellow line. So you can see .that some of that was occurring .during that period of time. It .was the second most common. We .also had new registrations .during that time, which was a .big chunk of what was happening..The interesting thing about this.data, .though, is that that is per VSPC.during that period..So we were looking at around two.or three changes or updates that.were .occurring in a voter service and.polling center during the first .week and into the second week .prior to the election..So very little activity .honestly. .So we used this to talk to .legislators and important .community activists on elections.and describe for .them the need to change the way .we do .our VSPCs to sort of roll back .the number and length of time in.which we needed to be open .because very little activity .occurs in those first couple of .weeks, or certainly in that .first week..We were pretty much wholly .unsuccessful in that .conversation because people .really like the fact that they .were open for that period of .time regardless of the fact that.they weren't actually being .used. So we were paying for a .lot of judges .to be there when, in fact, .nothing was actually happening..But when you have conversations .about that, they have kind of .other concerns which they're .maybe interested in, which are .more national in perspective, .which frankly aren't all that .related to the actual data that .we were able to produce for .them. .>>> Another thing that we want.ed to talk about related to .voter .registration is using data to .sort of reduce county mailing .expenses. We'll hear from Amber.McReynolds later, who's back in .the corner. Wave to everybody, .Amber..Another thing that we wanted to .talk about related to voter .registration is using data to .sort of reduce county mailing .expenses. We'll hear from Amber.McReynolds later, who's back in .the corner. Wave to everybody, .Amber..She and the Denver relations .team encouraged the use of .change of address .-- in fact, we passed the bill .in 2013 -- which led to the .automatic updating of voters' .files based on national change .of address. So if we received a.notice that .somebody had changed an address .from NCOA, the county could .automatically update that .information. They didn't have .to ask first, which when you ask.first, you get a lower response .rate, actually a substantially .lower response rate. If you .just update it and then tell .them, hey, we just updated your .information. If you don't like .that, let us know. You get to .do a lot more of that work .because you get very few .responses when you actually .invite. .So you can see, based on that .activity, that we've actually .seen an extraordinary reduction .in the number and percentage of .undeliverable ballots. The .election that's missing there is.2017. We didn't actually have a.statewide election. We had a .coordinated election in which .counties had issues on the .ballot, but there was no .statewide issue. .Our NCOA numbers for 2017, most .of them are intracounties. So .these are people that are moving.within a county. And then .intercounty, so moving from one .county to another county. And .then our out of staters. But .for the law allows us to .automatically update, .automatically change the .inter-county and the .intra-county. So over 75 .percent of the notices we .get from NCOA, we can make an .automatic change to that file. . Here's the NCOA .numbers across months. Again, .this is useful data for helping .counties to better staff up or .times in which there's going to .be a lot of activity and .corresponding vacations and .so forth around the times in .which there's less activity.. So what has automatic .update of NCOA done for us?.In 2016, 152,000 in-state NCOA .updates were automatically made .by county officials. Each of .these saved one mailing at .least and a reduced number of .subsequent failed mailings, the .regular mailings and the ballot .mailings, which are expensive .because of the rate of return. .If those hadn't been updated, we.probably would have seen less .than 10 percent of response to .those mailings asking and .encouraging people to make that .update..So 90 percent of the address .changes that we ultimately would.have had to .make at a VSPC, at a polling .place during an election cycle, .we could make prior because of .the NCOA information. .So savings..We see these as three major .areas of .savings -- the reduced mailings,.the .automatic update, which reduces .our mailings, and fewer .in-person updates..Counties probably saved $500,000.in .2016, and that was the biggest .election year recent ly..Using this data, Denver has some.data that suggests that number .is probably even double. But .that may be more specific to the.Denver model or the metro model..Certainly, many counties would .see an extraordinary savings or .have seen an extraordinary .savings using NCOA to .automatically update. .So those were my three topics. .I'm happy to answer any .questions when we get to the .question and answer period..>> Great. Thank you, Judd..We'll pass this over to Ericka..Next up is Ericka from ERIC.. >> Hello. Yes, I have.the most appropriate name for .any worker ever. So I am Ericka.from ERIC. I have a very long .title, which basically means I'm.the geeky person in .our business, and I get to play .with data a lot. .So I don't want this to be an .ERIC pitch, but I do want it to .be a case study of what states .can do when they join /TOLGT. .Together. We're a little unique.in that we have quite a few .states doing something for the .same purpose. We try to make it.broad in the sense of hitting .angles that appeal to red .states, blue states, purple .states. We are across the .nation. Darn, my new slide .didn't get in there..This is 23 states, but we were .happy to welcome South Carolina .two days ago. So their data .isn't in our fold yet, but we .have welcomed them into our .membership.. Oh, I got the picture,.but I didn't update the title. .ERIC is a member organization. .It was created by the states for.the states. I answer to the .board of election directors..We have the 24 states, which now.with .24 equals about 39 percent of .the voting .eligible population, according .to the 2016 members from the .U.S. election project. We were .created in 2012, but that was .actually a process that started .back in 2008 in a conversation .and took several .years to build because it's a .very purposeful, methodical .process that was well thought .out in the sense of how do we .make this work and make it work .for our states?.And what we do is we take in .voter registration data and DMV .data, and we will be taking in .other service and agency data. .We take in information about .voters and DMV, the contact .information..So the top, their full name, .mailing address, residential .address. Their information .about their activity because for.us it's finding out who has the .best data about a person. So .activity dates and status. We .also take in -- and I'll talk .about .this a little more later -- we .take in data that we .purposefully hash. So I'll go .into March depth on that, .but that means we make the data .so it is not -- we never receive.clear text of .date of births, driver's .license, or ID number and SSN. .That's a way to protect the data.but we use it for matching. We .purchase the death file from the.social security administration .so we're able to process against.the deaths across the nation, .and that's especially helpful .for states when a person has .passed away outside of their .state. .And then we purposefully exclude.protected or confidential data. .So that's never in our system. .So that can be protected at the .source. . So we take in that .information from our member .states, and from that we produce.reports back to our states. So .we only communicate with our .state elections officials this .information. So we take a .two-pronged approach..We're doing outreach to do .eligible but unregistered .mailings. Currently, we have .DMV and voter registration data..We do intend to have other .service areas..But the idea is to find a DMV .record that doesn't have a .corresponding voter record, and .those are people who are .potentially out there that could.be registered and states can do .outreach to. On the other side .of the house is the list .maintenance, and that is finding.-- by using this big matching .engine, which is basically what .we are, a big matching engine on.steroids. We're able to find .instinct moves, and that's .actually our largest number even.though across state that's what .people think of us about. But .in-state movers, just as Judd's .data suggested, people move .closer to .home more often, and that's just.as .detrimental to their voter .record as if they moved across .the country. . So in-state is when .the DMV has more information .about where the person moved. .Cross-state has another record .that's more recent, so it would .make it appear the person moved .out of your state..As I mentioned, the social .security .death mat /* master, so matching.those records. We do matching. .That's what we do. But also .because we have so much .information coming in -- and .that's the basis for our .matching engine..It's a product from IBM /* IBM .and our product company sensing..So when you have something new .comes in, you can find the .person. The social security .death master is really .interesting because many .election folks, many voter .registration systems, you take .either the driver's license or .the SSM..So in many cases, there's only .about 20 percent of the records .that have a social security, .last four digits on them..So if that local agency was able.to purchase the social security .death master on their own and .match it to their system, .there's 80 percent of the .records that don't have a last .four SSN they can even look at. .So you only get name, date of .birth out of that..So it's harder to match to. .One of the things we like to .point out is that we approach .security from the ground up. So.when we were building our .system, .when we were con receiving /* .receiving it, we talked about .security and how we protect this.information in our care. We .approach it from a standards .based management practice..We look at the ISO standards and.kind of develop our standards .around that. We do risk .assessments yearly, and we're .thoughtful and methodical about .how we approach security. We .also take an approach of .continuous improvement. And I .think you'll probably hear this .in a theme throughout the .conference if you're at NAS. .We're never done with security. .We're never like, okay, we got .this. We're ready. Because .there's always the next new .thing. So we're always .improving. We're always looking.at what's out there, how we're .changing, and how we need to .change with that. .We approach with both internal .and external reviews. We'll .head into a security audit along.with financial audits. Those .are important things for us. .And I have the picture of the .onion here because that's really.the most important thing for us .about how we approach security..There is no single piece that is.infallible. When you were a .technology, security person, and.cyber security especially, .there's no piece that has no .vulnerabilities, even if it .might be infinitesimal. So we .have a different layer around .the onion that protects the weak.spots of one with the strong .spots of another. So that's an .important concept for us. . I mentioned that we .protect the three specific .pieces of data -- the .date of birth, the last four .SSN, and the driver's license .number -- but we use a crypto .graphic one-way hash, and that's.just a big mouthful that .basically says the information .goes through a process of .hashing, which makes it into an .output here that has a jumble of.letters and numbers which aren't.helpful to a person who just .wanted to look at the data. So .I can't tell a person's date of .birth in any piece of data in my.system, but I can tell does this.date of birth match another date.of birth? The same jumble of .numbers comes out of this .hashing function, but it's only .useful for matching. So that .hopefully, again, one layer of .the onion is this makes the data.uninteresting for someone who .might have a nefarious purpose. . And what's important .about .this being one-way is we never .intend to decrypt that .information. We never intend to.have it in clear text. So it's .a one-way hash. . So some of the things .that are .interesting to me at least, as a.data geek, because I get a .different view on this data. I .get to see lots of data come .through my hands. It is me at .the moment. I am the person at .the keyboard looking at this .information. When we're .onboarding, I am a new set .of eyes on the data out in the .state, and sometimes they find .that interesting, and sometimes .they don't because we're looking.at very specific data points. .So things like activity date, .which is trying to find out when.a person had an interaction with.your agency, both voter .and DMV, that was meaningful and.from that person..So we find things like how .county practices may differ .across the state. I get to see .things like list .maintenance, the activity over .time. I take information .monthly or every two months and .compare it to the last. So if I.suddenly have a large number of .records that are different, .generally -- either there's an .election, which is easy to find .out. If not, I reach out, and .generally there's a list .maintenance activity that's .happened, and I can kind of see .that unfold in the data. . As these data points .mentioned about elections, the .other thing, I went looking to .see, well, can I kind of tell .how people move? And is there .more of a constant rate? You .really see it articulated how .voter registration, we have this.big swell, and then it kind of .falls off as the people move and.the registration, you know, it's.harder to get that information .updated. Versus DMV is such a .steady rate. So seeing that .comparison is kind of a .fun thing in the way I get to .see data. .How addresses are entered .differently across different .states. But mostly I get to see.the comparison of DMV versus .elections, and things like how .you might deal with a person who.is homeless will be very .different or people who are .continuous travelers who are now.living in their RVs as they .retire and are exploring the .country. .Also, it's been very interesting.to see how DMV practices differ .in different states. Things .like in one state it's very .common for a person to have a .license and an ID card. So .there's always two pieces of .information versus other states .where you only have one and one .ID throughout the lifetime of .your interactions with the DMV. .So I get to see those kinds of .things. . This is just kind of .an interesting way I look at .data. I thought maybe you guys .would find it interesting as .well. I went looking for .someone who moved around. You .can see this progression of they.were in Wisconsin in 2011, in .D.C..in 2015 to '13, Minnesota, .Maryland, Virginia. And this is.-- while it's generic .information, this was a real .person's kind of progression .through. And what I find .interesting, and part of what's .core to our system is you'll see.the mailing address from .Wisconsin to D.C. It was their .mailing address in Wisconsin. .It became their residential .address in D.C..They're using their phone number.across multiple states..That's really common, so phone .number has become much more .useful for matching. And also .e-mail. People are keeping the .e-mail addresses longer and .longer. That's a way to find .someone and to reach them. I .just find this interesting. .Maybe you do too. I'll pass .that on to Don..>> Thanks, Ericka. Don, you're .up..>> Thank you.. So thank you. It was .a privilege to work with ERIC in.Virginia bringing that to the .Commonwealth. I think that's .very valuable too, particularly .as a statewide official..One of the things that we, I .sort of looked at, is what power.tools are available to election .officials because, as we'll see,.our challenge is keeping pace .with the voters to maintain the .voter rolls, and the .expectations are they sort of .anticipate that we will be able .to follow them, but they don't .necessarily notify election .officials..So we have to find technology .and new .ways to identify their moves and.then take appropriate actions. . The goal is to be .smart in how we do that and .accurate and focused on those .group of folks. Obviously, I .think, from my perspective, the .notification to the election .officials is key. If we can get.that information with some .confirmation, either through DMV.or .other sources, that helps the .process move along. .The challenge is upgrading our .processes to the latest .technologies and available data,.and that's sort of what we've .heard. So there are current .technologies and data that's .available. There's a tool for .everything. That's what I .learned growing up..The major sources of data that I.saw and have worked with that .have been very effective across .the country and what election .officials rely on is the .department of Motor Vehicles. .It's a database that's great. .It can either confirm what you .think you know. It can be an .investigatory database, and I .saw that in Virginia and .Florida, where you can confirm .perhaps a .potential registration or a .discrepancy or move. But if you.have access to the DMV on a .macro or individual level, it .helps resolve tough issues in .investigations. .Social security administration .and department of health and .vital statistics in each state, .that's been a very valuable data.on deceased. And the state .voter registration system and .election officials, when someone.moves inside of state or outside.of state, the local election .official will receive .notification, or the state will .do the same. .And the courts have been a .source of data. But what we'll .find is that their /TKEUR as .we'll mention in a moment, NCOA .is a very valuable tool, but .only about 50 percent or 60 .percent of individuals notify .the post office. Therefore, .there's a whole swath of .individuals we have to work with.to try to find what their new .address is. .And that's where the power .technology is. I was always .taught there's always -- power .can solve any issue. You move .from the basic tools to the .power tools, and, yeah, it will .get solved, right? We've been .relying on return to sender and .post office information, and .it's become a very valuable .thing. The vendors have done a .lot with the information. The .post office service has done .much, .and I'll talk a little bit about.that .and dealing with the vendors and.the .post office information under .NCOA. .There's other information that .these vendors can provide that .can be very helpful to identify .those who haven't notified the .post office..And also there's been some .events with EVVE, electronic .verification of vital events, .where some of their information .is very, very valuable to local .election officials or state .officials who either want to .investigate ones that are tough .to verify, the deceased status, .and then there's the third party.credible data providers. These .are companies that they work .with other agencies, federal, .state, and major companies in .the United States, and more and .more with election officials to .provide that little extra you .might need to resolve the .problem. . So, obviously, most of.you know what NCOA is. We've .talked about it already..Obviously, it's a national .database of 160 million records .of individuals that voluntarily .confirm to the post office that .they've moved, very valuable .information. What I found .interesting, though, and I felt .it would be good to share, is .that more and more this .information is available. And .those that don't provide the .post office, these vendors are .able to provide you that address.where that individual is now .residing..It allows election officials .some flexibility.. What I found .interesting is a lot of services.provide realtime access to a .user. Could be local or state. .I know a lot of officials who .use that, where if there's an .individual that you're .investigating or a batch that .you want to do, that you want to.process, you can do that in .realtime, and that can be very .valuable to a local office..Again, I just wanted to .highlight that, when movers fail.to notify the election office, .you're able to get that address,.and then you can send a mailing.. What I also found with.NCOA, just going a little bit in.depth and receiving some reports.from the vendors, is they can do.a lot of audit features..They can actually audit your .mailing .list and provide you some issues.that are very useful to you in .cleansing your rolls. And then .they can also provide some .analytics. Obviously, .everything costs a little money,.but the fact these are offerings.can really help an office. .Think about EVVE..It's a national nonprofit, and .they .basically are a consortium of .states' vital records .departments, the thing that's .interesting is they receive the .data directly from the state .death databases..Again, I was very intrigued by .the fact you can have large .volume requests and single .queries. A lot are using social.security administration death .master file, as .discussed, but this is becoming .a more inclusive record as we'll.find out in a moment. Again, .you'll get an electronic .response. If you have certain .individuals you're not sure of .their status and believe they're.deceased, you can actually do an.electronic search and find this .out. .As you can see, more and more .jurisdictions are participating,.and they claim that they .actually do have more records .than the Death Master File .because there have been major .changes with social security and.the Death Master File. A lot of.information is no longer .available to election official, .and .they're actually making it a .little more difficult for .vendors to provide that data to .the states and to localities. .So I thought it would be .interesting to bring up EVVE .because they don't have some of .these same issues. .So I thought it was very .interesting because they are -- .they .have more records than the Death.Master File because many of .these are not being released to .the public anymore because of .protected status. . EVVE has a growing .number of clients, both at the .state and federal .level, including department of .motor vehicles and the secretary.of state offices..So it's become a valuable tool. . Credible third party .data, public records..You know, some of the players in.this are LEXIS..Many of you have dealt with .LEXIS or Experian. These are .multibillion dollar companies .with tens of thousands of .employees, and they provide all .types of information to folks .that they have a financial .interest in making sure the .address is correct and the .identity is correct. So they .have an interest -- the .information is fairly accurate. .So what they do is they do .provide .these services for states and .locals to .do person address locators for .that last mile, trying to .identify those that haven't .identified their move to the .post office..Their engines, they have all .types of military locators, .voter registration records they .actually collect from the .states, and you can see all the .records they have, and they use .this as part of their services. . They can provide .address cleansing to your list. .And really the goal is to .actually reduce the amount, as .Dr..Choate mentioned, bad addresses..It helps cleanse your -- it .saves on undeliverable mail..And you can see all the records .they use to provide that service.to their customers. . Neal Kelley did a .study back in 2012, and he's .continued this. To talk about, .he wanted to do a study of how .this credible third party data .would work for his office, you .know, a major county obviously .in California. As an initial .test, what he wanted to do is a .nationwide service. He sent out.a thousand names of individuals .that they had no activity and .they believed that move, but .they had no notification from .any source -- the NCOA, DMV, .election office. They had no .other source of data..So they did this test and sent .it out to 1,000 individuals, and.over 500 came back with an .accurate address. .So they tested this in -- what's.interesting is that 280 of them .had moved at least five times. .It shows you the mobility of .voters and that sometimes these .are the hardest voters to try to.keep up with. So then they went.to a larger study, .and they analyzed 250,000 .voters, and they were able to .update addresses for 122,000 of .them. They were able to send .addresses to these individuals, .informed them they should .register in their new state or .jurisdiction if they had not .already done so, but also give .them the opportunity to update .their address with his county. .Over 18,800 responded to the .postcard, and he found just with.this one mailing, .he almost was able to find .savings that matched the .expense. . What's remarkable, .though, is it only took two or .three elections, and obviously .the investment paid off. He's .told me that he's done this -- .and he'll be here later today, .and you can talk to him. He's .actually done this three or four.times since 2012, and it .continues to be a practice in .his office..It's major savings on postage. . >> Great. Thank you .so much. So we'll move to some .-- thank you to our panelists. . [APPLAUSE] . I'm going to move to .some of my own questions, and .then you can be thinking of your.questions. I think this is kind.of a general question for all of.you, so we can kind of go down .the row..HAVA provided funds to support .the establishment of .computerized statewide voter .registration databases, and some.of the new HAVA funds are being .used to .update those databases, so /* to.strengthen them, especially when.it comes to cyber security. . I want to ask you all,.what is the state of these .databases? Are they healthy? .What are the kinds of things we .can do to improve them?.Including, you know, of course .the cyber security. Let's start.with you, Mike, and hear what .your thoughts are about the .state of our databases.. >> I think the state .of the .database, specifically in .Pennsylvania, it is in good .shape. We do a lot of testing .of the database..We have some very dedicated .teams in close collaboration..I think some of the challenges .that .we're seeing is.more of the reporting .capabilities and more of the .flexibility of the database and .getting more information that .would be easier to take out of .the systems..Always with the benefit of .hindsight you see it differently.when the database was first .implemented back in 2005, 2004, .you can easily say these are the.things I wish I had, looking .back at it. And I think a lot .of it has informed us to get .better metrics along the way of .how applications are being .processed and the way election .results are .captured, things of that nature.. But overall, I think .that we .have pretty robust cyber .security layers in place..We work actively with our office.administration and other .entities in the .state, like FEMA, and forging a .closer relationship with the .national guard. Certainly, the .U.S. department of Homeland .Security, we've done a lot of .work with them to make sure the .security layers we have in place.are good and make sure the .system is secure. We're really .focused on making sure we .can leverage newer technologies .to really get better insights on.how .everything is being processed .within the system to make sure .we can continue to go down the .path of making more data driven .decisions to meet people where .they need the services.. >> Judd, what are your.thoughts?.>> In Colorado, we have the old .system from sabre. Sabre is a .company that hasn't been in .business for ten years now, so .you can tell it's kind of an .older system. But I think we've.made pretty good decisions about.it..Most notably, we hired several .of the programmers and .developers of that original .system and brought them on .staff, and that's allowed us to .keep that to be a dynamic .database where we .can always evolve with new code .to make .it more useful going forward. . We do have a current .project right now, which we'll .support through the new HAVA .funds, to roll out basically a .web version of our score system,.and that web version we can .enact a lot more security .measures.and make it a much more user .friendly database..Right now we're linked to a .Citrix .internal network to make it .work, and our future we'll be .expanding behind that to what we.call score 3, which is sort of .the third version of it, which .will allow us to be much more .sort of user friendly and .dynamic. .So that's the future, but we are.in a pretty advantageous .position because we have code .writers on staff that know the .system. And so they can .literally make that old system .function like a new system. . We do foresee the use .of HAVA money for a number of .different security upgrades, .including firewalls and .tracking more -- in a more .minute or a microlevel the .actual activity that occurs in .our database. So it's easily .retrievable. So if we see .something, we can jump on it .immediately instead of perhaps .hours .or a day going by before we .realize that something's going .on..So we're using that new HAVA .money for .tracking and improved security .with firewalls..>> You've got an interesting .viewpoint, seeing that you can .see a lot of different databases.from a lot of different states. .What are your thoughts on the .state of our databases and what .we can do to improve them?. >> Well, I think it's .definitely, as you mentioned, .hindsight is always 20/20, but .we've learned a lot, and I think.that's probably the biggest .important thing for what I see .is our elections officials take .this seriously. I think we all .know this in the room. But .getting to learn about their own.data and what's out there, that .we're more interconnected than .ever..Things like ERIC, where we have .24 states that come together .regularly, they talk about .stuff, and they see each other's.practices. . Like I just got to .help Oregon and Minnesota have a.little demo where they're .checking out processes on how .they handle some records and get.to learn from each other on .those things. So I think that's.encouraging, and I think that's .happening more and more. And I .think that helps improve the .security across the nation and .the state of the registration .databases. .As I onboard states, I get so .many more interesting questions .now, and .I've gotten to /* to see the .progression of states, our .original states, and the data, .how it's progressed, Judd could .speak to how they got to compare.to their DMV data and find it .lacking, and also changes in how.they handle addresses and more .consistency. More states that .have been our bottom up, where .the counties still have .individual pieces. When they .use something like ERIC, they .get to see a global picture of .their own state and how they .might be using processes and .using their systems..Even some of them in their top .down, how the counties actually .use their systems. I think .there's more tools and more .opportunities, and our elections.officials are taking them and .taking advantage of them so they.can improve..So I'm encouraged.. >> Don, do you have .something to add?. >> I would just add .that the statewide voter .registration systems is .really a service from the state .to the localities..In this business, there's not an.option. You have to be .accurate. You have to provide .those services. And what .happens on election day in .early voting can matter, .obviously, to voter confidence..And so if you have a 10-year-old.system, how confident are you .telling, oh, everything's going .to be good. Fear is not an .option. It takes a lot of work,.and it's always good to have a .continuing upgrade .of your voter registration .system because you want to make .sure going into .early the /* voting election .day, you .want to fulfill those services, .to election officials and to the.voters, and it's obviously an .evolving process. But I think .there's areas where we can .upgrade our systems..>> Privacy is a big concern .for voters when it comes to .registration data..And I just want to hear from you.all .what's being done to address the.privacy concerns. Are we doing .enough? I get this question a .lot from voters who are very .concerned about the privacy of .their voters. Let's start with .you, Judd.. >> Well, obviously, .privacy is a huge concern, and .that's a lot of what .you're doing when you're .protecting your database..One of the big things that we .try to identify is various ways .in which somebody could .penetrate our system. We've .done the risk and vulnerability .assessment with the DHS, where .they've come in and basically .lived in our office for a couple.of weeks and tried .every various way to try to get .into our .databases, both those that are .elections related. But also we .have business filings and .charitable filings and many .other services that we provide .on the business side of our .office..So DHS has really been effective.in that. .But all of those are about .protecting our voters and our .constituents' information .because, if we .don't protect it and it gets out.and is available either on the .web or on the dark web, it's a .violation of all that we are .trying to do to protect it..>> Mike, do you have something .to add?.>> Yeah, I'd like to echo Judd's.comments..We are very much concerned .>> Mike, do you have something .to add?. >> Yeah, I'd like to .echo Judd's comments. We are .very much concerned, and that's .one of the first things we're .thinking about when we're .putting a new build out or .getting information from the .voters, we want to maintain the .privacy of all sensitive .information as best we can. . And that's why we .really focus .on the routine scans that are .out there..We also have physical testing .that takes place as well in the .office to see if there are any .exposed ports..And we have really renewed the .interests of seeing what the .practices are for not only data .in transit with .our business, but also data at .rest to .make sure that we are meeting a .lot of .the federal standards and our .Commonwealth standards on data .encryption and protection..So we really do try to put as .many layers as possible with .firewalls on the database and .logging of networks and the .things of that nature to make .sure that nothing is getting out.there that shouldn't be getting .out there and that voters can be.confident that their data is .safe in the database..>> Ericka, you talked about .hashing the data. That was .interesting. What are your .thoughts on continuing .to make voters' information .private? .>> Well, I think again -- and .the questions that I now get as .we have .new and prospective states, I .think the awareness is so much .higher. That's kind of on the .front of our elections .officials' minds, and I think .they're becoming more and more .educated. They ask us pointed .questions about how we protect .data, and I think that's of .critical importance to us. .We're more and more aware of the.tools that are available. .Again, that interconnectedness .of us talking to each other .makes us stronger as a whole .sector, and I think that's .happening. . So I think that's .where we continue to grow .through..Things like our tools of how we .hash data. I joked that we're .trying to be purposefully .uninteresting..We want to be boring and not .looked to as a source of data .they can use..I think the awareness out there .for .others is, as our election .administrators, they're more .educated and more aware than .ever..>> Don, I remember when you were.in Virginia, you had some .litigation over this issue, .right?.>> Well, we did. And part of it.is I think part of the initial .attitude is that we're here to .protect the data of our voters. .There are -- I mean, there are .legal mechanisms for the release.of data, not .only FOIA, but the legislature .has -- the legislature and the .political parties usually .receive a list of registered .voters. So their mechanisms in .the state where the data may be .released..And the courts have basically .said also that certain .information is eligible to be .released for persons that .request it. .So it's weighing the balance .between open ness.of our process versus the .privacy of voters. It's .balance. So we face that in .Virginia, yeah..>> I think you said earlier the .judge had provided an order with.a lot more open data than you .expected..>> Well, it was interesting. We.had a case where the order .initially basically was going to.release voter registration .records, and then .there was sort of a modification.because they would have released.social security numbers and .other data. So the judge .modified the order to protect .certain data from release to the.public just generally.. >> So I'll just do one.more question before we get to .the audience..I just want to ask about the .recent .Supreme Court decision in.Husted versus Philip Randolph .Institute. And the action in .Kentucky has brought attention .to the list of voter .registration list maintenance. .How do you see list maintenance .practices changing in light of .these developments? Don, why .don't you go ahead with that? .We'll kind of go down the line.. >> Sure. I think that.-- and it's one of the .reasons I focused on the area. .I think there is data out there .that allows us to focus our .resources. So it can supplement.and sort of .refine sort of our mailing .processes for NVRA. Obviously, .you can submit mailings to all .registered voters. The court .opinion was directed at can you .focus a mailing on those who .haven't voted in two, four .years, and then start the NVRA .process?. I think that part of .my presentation was to talk .about the tools .that are out there to make -- to.refine .our mailings to those that we .absolutely know have some .indication, some notification .that they've moved. So you can .do smart list maintenance with .mailings and with targeted .information data that's .available. .>> I feel like I'll end up like .a broken record a bit. What we .find, because we're all about .list maintenance and outreach, .but list maintenance happens .more regularly across the year, .is our states are comparing .notes and finding what works for.each other and taking that .information back once they've .talked to us. I think that's a .really important thing. So .they're finding out what works, .and .then we see it play out in the .data, and that, again, having .data that you can look back to .and actually taking the time to .find out what worked and what .didn't and how you can modify it.is super important. . The other thing for us.is we try to be very focused on .actionable data because there's .so many limited resources. So .making sure that you have what's.in front of you, things you can .act on succinctly and not have .to sort through a lot. So .that's important from our .standpoint.. >> Great. Judd, we .saw each other on the day of .that argument. It's the day of .our 2018 summit, and you went to.the argument..What are your thoughts on the .Husted case?.>> I was very fortunate to be .there..It was the first time I had seen.a U.S. supreme court argument. .It was a very interesting case..I think the ruling is basically .an earthquake in list .maintenance under the NVRA. But.ironically, probably won't have .all that much in the way of .long-term consequences because .it is a complete shift in the .way that we see the NVRA and the.way we see how you can change .your record..I think it was a pretty widely .held belief that you couldn't .really do .anything with somebody for their.failure .to vote, even if you sort of .padded it with additional .protections like doing a .follow-up mailing, and that was .a widely .held belief in the elections .community. .But the Supreme Court, in the .5-4 decision, decided that was .not the case. However, it was .really only a handful .of states that were doing the .kind of Ohio related practice, .and even Ohio really doesn't do .it anymore..They have joined ERIC since .then, and a lot of their list .maintenance activity .is happening through the use of .ERIC data..So this isn't -- this is a .practice that likely won't grab .hold in the elections community .because I think we're all .looking at better information .than this because this really .was sort of a machete that .states were using when now we .have scalpels..So it's not really -- it likely .won't .have that kind of long-term .consequence..>> Obviously, the NVRA was .passed in 1993 before we have a .lot of the tools we have now. .Mike, do you have any thoughts .about that?. >> I agree with all .the general thoughts up here..Specific to Pennsylvania, I .don't think we see a direct .impact to what we're doing..We're looking to continue to.leverage the robust reports .coming out of the ERIC program. .It's been a great program, and .the actionable data has been .very helpful. We don't see a .direct impact to what we're .doing today because we kind of .go through a normal five-year .vetting process as it is and .have plenty of time for that to .play out. . >> So let's go to the .audience. Do we have roaming .Mikes. We have one there, one .over here. Got a question here..Remember Tom's three things -- .speak into the Mike, tell me who.you are and where you're from, .and ask a question.. >> Hi..My name is aThen from Bucks .County, Pennsylvania. I have a .question for Mr. Moser. In the .presentation for the online .voter registrations, you had .reference the third party API. .Can you talk a little bit more .about that. Who can access .that? How does it work?. >> Great. Thanks for .the question. Essentially, the .way the process works .is it's an entity is interested .in conducting a registration .drive and .would like to build their own .online application, they can .essentially go through an .approval process with the .department, and they'll sign up .for an account, and they need to.go through a terms of use and .agree to terms of use that's .publicly available on our .website and need to go through a.series of application testing to.make sure they're adhering to .our standards and that they .have a lot of the pertinent .information there for .registration, and they're .certainly meeting all of the .legal requirements for voter .registration to make sure that .information is there. .So we engage through a testing .process, and then they .essentially are .assigned a unique key on the .back end to add security layers .to it and to help us track the .information that's coming into .the online interface so we can .continue .to monitor metrics with the .entities that are submitting .application for not only .compliance, but to make sure .we're not running into any other.issues. .It's been helpful to track the .registration drive process, and .I think we've been seeing .benefits in the short .time that it's been up for over .the year, year naphtha we've .been able to reach out to .entities and kind of coach them .along with best practices as .we're seeing information coming .through, but they've done a .really good job adhering to the .testing requirements and the .security requirements and making.sure they're following the .registration guidelines.. >> Colorado has an API.as well. We've got one over .here. Microphone is making its .way.. >> Hi. My name is .Sheila Laney..I live in Montgomery County, .Willett Grove..My question is regarding that .supreme court decision. I think.it has to do with intent..Like when I listened to your .panel, I'm really hearing like .somebody .mentioned the geekest part of .elections and the processes. .But when your intention is to .not .allow people to vote, then that .changes the whole mix..The question I have is a couple .years ago there was a purge, and.I think it .was Bucks County, of 14,000 .voters..So I'm listening to the .processes that .it takes to maintain a valid .list, and the money that Orange .County did to go .through that whole process, what.here in Pennsylvania are we .doing so that we don't have to .purge people because they're .either not voting, we can't find.them, the addresses are not .connecting?.The second part, as a .community person, that online .voter registration form, is that.available now? The updated .version?.And also, is the paper version .updated? Is that available now?. >> So Mike?.>> So I'll start with the last .part, and thanks for the .questions..So the updated paper form is out.there available today..It's listed on the votes PA .website..We also have a few forms in .several .languages there in the same .format. .The online voter registration .application is available, and we.have the third party option for .registration drives. That's out.there today as well..So both of those are accessible..And I'd be happy to talk to you .on the side and can give you .direct links for that .information. . The other information,.I'll say up front, I'm not .familiar with that .specific instance, but what I .can talk .to more globally is.that, one, we don't ever purge .voters here. There is a process.that, through the list .maintenance practices, that .they'll be getting notices to .try to get accurate information .of where their location is or .inactivity, and we go .through a pretty extensive .vetting process is time for a .response. The individual still .has the .capability to show up to the .polling place to go through an .affidavit process .to bring them back up into .active status. .But we're really trying to focus.on that area. It was a really .good question to bring .up is where we really want to .not only take advantage of more .actionable data for current .records, but part of the reason .why we're putting a lot of focus.on modernizing the voter .registration process, and what .is going into that is clarifying.a lot of the language that's .out there and conjoining that .with the information that's .actionable on where .someone might recent to make .sure they understand with the .noticing that are coming out and.by collecting information .through the online forms, .designing them in such a way to .get more information, like an .e-mail address or a phone .number, is we really want to .start building a more solid .foundation to have a more .interactive process with that, .to make sure the individual is .aware that we're looking for .clarification on, say, like a .change of address or something .so they can respond to the local.county authority to mitigate .potential issues like that.. >> Judd, do you have .any thoughts about maybe .cleaning lists and the .maintenance of it?.>> As an elected official, I .hate the word purge. I hate to .use it..The only time Colorado does a, .quote, unquote, purge would be .after a federal general. These .are people who two cycles ago .got a mailing, got a .confirmation mailing that .bounced. It was undeliverable..So we know that they don't live .where they were registered..So then they don't vote in the .next election, next federal .general election, and then they .don't vote in the next federal .general election. So typically,.that's a four-year window. It .could even be more.. When you have that .assurance they don't live there .and they haven't voted in any .election, they haven't had any .activity whatsoever, and they .didn't vote in the two major .elections, one of which had to .be a presidential election, .at that point, under the MVRA, .you can cancel somebody, and .that's the voter purge that .happens in January, February, .and March typically..That's the window after a .presidential or after a -- .either presidential or .congressional election..So we'll have one in January, .February, and March of 2019. . But that's really the .only time you should be doing .that kind of level of list .maintenance where you're .cancelling people in buckets. .Otherwise, you're just using .information about somebody who's.died or has notified you they've.moved to .another state or in the larger .case, where you've had data upon.data upon data that they don't .live there, and now .you've gone through a two cycle .process..>> Any thoughts, Ericka?. >> List maintenance is.so imperative to what we do. .For us, it's all about trying to.get information in the hands of .our elections officials that can.fix it..So for us, the DMV is such a .great source because people, .when they move, they go to the .DMV..And many of them think that it .automatically flows through. In.some cases, it does. There's so.much more interaction with the .DMVs than it was even five years.ago. That's part of why we .exist is to try .and make sure that elections .officials have the information .to update the records and get it.so the folks don't even have to .really notice almost. There's .mailings and things that go .along with the processes we .suggest to our states, and each .state does it a bit differently..But that's what we're all about..>> Don, I know that DPC has been.doing some line work..This is kind of related, isn't .it?.>> Well, it is. I think that .the collection of line .data, another issue that the .country is faced with lines on .election day, and .BBC, where I'm a fellow at, we .have focused on the collection .of line data in jurisdictions .across the country. It just .goes to show that data can .actually inform us on the .allocation of voting machines, .electronic poll books, a number .of stations. It can help .identify us to the local .election official, who's really .the person that's going to be .managing that process..So I thought I'd mention at the .data summit that BPC is doing a .lot of that data..And it just shows you how data .can be used to sort of improve .the process we're undergoing.. >> Clean data is .important to make sure we don't .have long lines, correct?.>> Clean data is very .important because, you know, .inaccurate .voter rolls or snafus can cause .long lines or long lines and .provisional ballots. It can .cause congestion at the polling .place, confusion, headlines..We can go on and on about where .that goes. It's never good when.you have inaccuracies in the .process..And so it's better to just make .sure .we get to a point where our list.can be as accurate as possible .when we go to the polls..So there's less confusion about .am I in the right polling place?.Am I in the voter rolls? And .those sort of issues that, when .they do occur, can cause .problems in a polling place. .Election officials and poll .workers have plans, plan Bs in .place, that these things do .happen, and presidentials. A .lot of people are showing up to .vote..So they have these .contingencies, but it's always .good to do as much as you can .prior to election data resolve .these problems..>> So we've got improved access .for data in elections, better .quality for data in elections .that continues to improve. I .think that's improving our .processes as election .administrators. I think the .next step, and the .question is what can we do to .share that with the public to .build public confidence and .share that with policy makers to.improve the administration of .elections from a policy and a .legal perspective with this data.kind of as the next step. . >> Which panelist are .you? I want all of them to take.a stab at it..>> If anybody's got it.. >> Don, you want to .go?. >> Sure. One of the .things I wanted to express .is data provides -- it's .agnostic..You can -- believe it or not, .both political parties and .politicians, when .they're faced with data, it .often does move the needle. .It's hard to believe, but it's .true. So collection of line .data or how we're doing on voter.registration or those sort of .things, you can bring that .to the decision makers, like a .city council or state .legislature or the federal .government and say this is the .data. This is where the trends .are..And I think that that serves a .valuable purpose in where we're .getting .-- the community is becoming .more -- there's more of an .ability to extract the data and .provide reports to decision .makers, policy makers. So if .you're able to do that, we can .move in a direction where we .continue to improve the process..That's my opinion.. >> Yeah. I mean, data.can help take emotion out of .some of these decisions, right? .It can be misused, but it can .also take emotion out of these .questions. Do you have .something to say, Ericka?.>> For better or worse, we have .more of the public's attention .right now because more of this .is in the news. So it kind of .is -- the time is ripe if we can.get those things. I don't want .to steal anything from .Amber, but things like what .Amber McReynolds does in Denver .County, where she produces .really consumable information .from the data she collects. I .mean, info graphics is a bit of .a buzz word, but it does really .make it understandable. So .where we have these data points .-- like I love that Pennsylvania.is approaching their next .iteration of their voter .registration date thinking .about what they want, what .answers they want to know first.. The things that we can.get out in those tweets -- and I.hate to even .say that, but those little bits .where we can get accurate .information from reliable .sources out while we have the .public's attention, the time is .right. It would be great to do .so. Maybe in somehow connecting.with interns or those folks that.are in that kind of graphical .information sharing to .tell our story, I think could be.really useful. .>> I'm going to go ahead and .take the prerogative to ask one .more question. Unfortunately, .we could go on for a while, but .we have an end point..My question is to Judd and to .Mike, since you all answered the.last one. State officials .across the country have .differing levels of authority .vis-a-vis local jurisdictions .and state and local relations .can be tense. In your .experience, what approaches can .states take to navigate these .obstacles and build the .necessary trust among local .jurisdictions to strengthen your.statewide systems for data .collection, analysis, and .sharing?.>> Well, so a couple of things. .Over the last several years .we've taken a multitude of .approaches..Right now with Secretary .Williams, who's in the audience..He can wave to everybody. .Secretary Williams, who's my .boss and who's been my boss the .last four years, we are very .much arm in arm with the .counties. We work very closely .with the counties. Every .decision we make, we are .involving the counties. . We haven't always been.that .way, and, in fact, we were even .suing .the counties on a pretty common .basis a while back..That can work. It doesn't .always work. If you have a very.specific issue that you need to .address, sometimes the courtroom.is where you can address it..But for the most part, I think .working together makes sense..And one of the things that we've.done in Colorado is we started .collecting data on elections .costs, and I think that that's a.pretty new phenomenon..We've been doing that for about .the .last five years, and we've tied .it to something that we have in .our law, which is that we .reimburse counties for federal .or elections where there's a .state question for -- we .reimburse them either 80 or 90 .cents, depending upon the size .of their jurisdiction. .And having that, sort of a .carrot and stick approach, hey, .we're going to give you money, .but if you don't give us your .data, we're going to have to hit.you with a stick. Sometimes .that can be kind of an effective.way to work together to try to .figure out how we can get the .kind of .data that we need in the timely .manner but in a way that doesn't.really make the life of the .county all that much more .difficult, and they get the .reimbursement. So there's often.ways in which you can work .together to create a data .sharing .and get people on board, which .is can have multiple ways, both .carrot and stick.. >> Mike, last word on .this one. And then we'll wrap .up..>> I would say that we are ./WORBLGing on trying to not only.maintain, but also build on the .coalitions and partnerships we .have, specifically with county .boards of election. working on .trying to not only maintain, but.also build on the coalitions and.partnerships we have, .specifically with county boards .of election. In the past, it .has probably changed with .administerings, depending on our.priorities. But this .administration has done a .great job, particularly under .the acting Secretary Torres, to .maintain conversations with .counties to have open .die logs, to maintain the .communication, to talk about .what's in the system, what .changes are we making to the .system, and what are initiative .that's are really important to .them that we might not be .having a top priority at the .state, and we'll readjust our .priorities because we .really want to maintain a line .towards modernizing the whole .process. . They have done a .tremendous job in sharing .information, and we have a .monthly meeting. We call it the.short advisory board..It's a member of six county .advisers that we have .communication with..We have meetings across the .state several times a year with .the different associations of .the counties and have an annual .meeting that we're there. But .we're also building new groups .out there. Right now there's an.election cyber .security working group that is .extending beyond the county .election office to .also the IT offices on the .county level .with state representation. . So we're really trying.to build off of past practice .and make it more effective and .have more .conversations about evolving .topics and a closer loop, I'll .say. And take that information .and try to help sway some of .those decisions with some of the.data that's coming out and what .we're seeing to just make .improvements that are great from.all sides of election .administration..>> So let's give a round of .applause to our panelists. .Thank you all. . [APPLAUSE] .I appreciate you all being here .very much..This was an informative .discussion on voter .registration. We hope that .you'll stick around for the rest.of the summit..We now have a break until 11:00 .for our next panel, and that .will be on election day .preparations. Thank you. .[Break ] . >> Good morning. Our .panel is just about to start. .If we could start to make our .way back to the tables. Thank .you very much.. >> If we can get .everyone to their seats, we'll .make our EAC .overLords happy, and we'll get .going. .Great. I'm Charles Stewart..Welcome everybody back to the .second session, both those of .you in the room and those of you.watching the webcast from the .beach, wherever you are..I'm sure that's the primary .audience today from afar. .We have an excellent panel today.to talk about moving on to .talking about election day..Being a professor from MIT -- by.the way, I didn't mention where .I'm from. I'm Charles Stewart. .I'm professor of political .science at .MIT and director of MIT election.data and science lab..Being the guy from MIT, I've .been tasked with letting you all.know about the WiFi situation..I don't know if they've actually.changed the paper on the tables,.but if .you look for the CCP WLAN as a .network, that's the network .you're looking for, and there .should be no password needed..Again, that's CCP, two Cs, not .three, WLAN, and no password .should be needed. So there you .go. .So let's get going to talk about.election day and election day .preparation. We have four .panelists who will be .talking about a number of .interesting topics. Just as a .way of kind of getting into .the topic, election day is the .day when.there's really no margin of .error..If an election official is not .prepared, you can't say let's do.it tomorrow. If an emergency .strikes, if something bad .happens, an option isn't to .delay things. The show must go .on. So election day .preparations are really, really .important. . My work as a political.science following election .administration, I got going, got.my interest in this area after .the 2000 election, which is true.of a lot of academics..One of the things that I've .learned over the last nearly 20 .years of studying elections and .studying problems .that show up in elections is .that .actually it looks like on most .metrics, .on almost every metric, whether .it be the quality of the voting .machines, quality of .registration systems, quality of.polling place practices, quality.of vote counting, election .administration .has gotten better undeniably .over the last 20 years. .There's a lot of -- there's .still a lot of work to be done. .There's pockets where work .really needs to be done, but it .is, I think, true, and I would .defend the proposition that .things have gotten a lot better.. One of the reasons why.I find this kind of amazing is .that elections .have also gotten a lot more .complicated in the process..It's not only that we have new .challenges because of security .concerns, because of the role of.technology, but also because .kind of election administration .has helped to complexify things .itself. In the 2000 election, .for instance, 90 percent of all .the votes cast in the .2000 election were cast on .election day..Now only 60 percent are cast on .election day. 40 percent are .cast either in early voting or .by the mails..And then election day voting .itself is complicated by vote .centers and other new ways of .voting. So things have gotten a.lot more complicated, and things.are getting better. .So how can you give greater .complication and yet improve .service overall? One of the .ways you can do it is by working.smarter. The other -- one of .the trends that I've noticed .over the last 20 years is an .increased professionalization of.the field..Bringing people into election .administration who have .technical backgrounds, who have .backgrounds outside, maybe in .logistics or statistics, in .areas where the job is to get .things done and to analyze the .situation and to work smartly..That's really, I think, the big .theme .of this panel is how do -- can .election administrators work .smartly in an .increasingly complex election .day environment and more broadly.is a more complicated election .administration environment. .So we're going to be listening .to some folks who have thought .about how to work in this .environment. I will just very .quickly introduce our panelists,.and then we'll get going..There are longer biographies in .the .packets you all have gotten, so .I won't read them extensively. . So my far right and to.your left, the first is Neal .Kelley, who's .the registrar of voters in .Orange County, Florida. Neal .serves the voters of Orange .County -- I'm sorry. Orange .County, California..I oftentimes give him the wrong .name, .and that's -- so it's an honest .mistake since I grew up in .Orange County, Florida..So a 3,000-mile mistake, and we .both used to have oranges in our.county, but we don't so much .anymore because of a mouse who .infiltrated both of those .counties. .Anyway, Neal is a registrar of .elections in Orange County, .California, .which is the fifth largest .voting jurisdiction in the .United States. Neal has held a .number of offices, .both at the state and national .levels .and has won a number of awards .and has .been recognized for his Sterling.business leadership in elections.over the years. .Next to me is Benjamin Uminsky, .the project manager, business .intelligence competency center .of Los Angeles, California..Ben joined the L.A..County registrar/recorder county.clerk's office in 2007, where he.worked as an HR analyst for six .years. He joined the .department's executive .office in 2013 to head up the .newly created data analytics .program, where he had been .working on various data science .projects involving revenue .forecasting, .whole worker classification .modeling, voter file .duplication, et cetera. He has .flown the coop. I'm good for .L.A..County, for other parts maybe .bad for elections. He's .recently left the registrar's .office to work for the L.A. .County Department of Health .Services, in their business .intelligence unit, doing many of.the same things he did for .elections. . Immediately to my left.is Melissa Frey, the special .assistant to the director of the.Pennsylvania emergency .management agency..Melissa currently serves the .Commonwealth of Pennsylvania as .the .special assistant to the .director of the Pennsylvania .Emergency Management Agency, .PEMA..She joined PEMA in December of .2016, where she's been able to .combine her unique background in.strategy development, finance, .as well as program .data and crisis management to .lead .PEMA's office of policy. . And finally, there's .Mark .Goins -- I never know if it's .goings or go-ins. So you tell .us.. >> Either is fine..>> We'll pin you down one day. .He's currently the coordinator .of elections for the State of .Tennessee, having been appointed.by secretary of .state Trey Hargutt in 2009, and .he's also past chair of the .EAC's standards board. Before .that, he served on the -- before.being the coordinator of .elections, he served on the .state .election commission in the.aughts from 2005 to 2009. .Before that, he was in the .Tennessee house of .Representatives. I just .discovered unfortunately he was .gerrymandered out of .representing Rocky top, .Tennessee, which is my favorite .song from the state and one of .my favorite places. But when he.was in the legislature, he .also served on the election .subcommittee. .We have an interesting and .varied panel. We'll jump into .the first presentation from Neal.Kelley. .>> Great. Thank you, Charles. .And thank you to the EAC and the.officials here from Pennsylvania.for putting this on. I think .this is fantastic. Thank you .for the invite. .So I want to focus on the use .of data for utilizing logistics .on election day and pre-election.day, and how we have used data .over the last ten years or so to.really inform the decisions that.we're making leading up to .election day and then looking at.the metrics post-election day to.see if that's actually working. . I'm going to walk you .through a couple of things here,.just in term of the prestory and.the history of how we got to .where we are today. So I took .over the office in 2005, and .when I did so, the booths we .send out to the field and the .polling places -- by the way, in.Orange County, we have 1,200 .polling places roughly, and we .utilize 12,000 pieces of .equipment..So it's a large operation, and .those would go out in caddies, .steel containers. So you have .eight booths in steel container,.and it was much easier to ship .them that way. That's the way .it was being done when I got .there, and I let a cycle go .through of that, and I thought, .this is strange. Why are we .doing it this way?. It really hit home .because in 2006, I had a senate .election, very contested .election, where we had very high./TURPout we didn't predict, and .we had lots of lines in 2006 .because we didn't have enough .booths in the polling places..So I wanted to relook at that, .and that's kind of the precursor.to where we started using data .in a different way. .So because of those lines, we .started to look at historical .turnout at the precinct level .and started trying to do some .forecasting at the precinct .level and then changing the way .that we shipped the booths .themselves. I have to give some.props to the brilliant mind of .Professor Stewart because we're .utilizing his tool now from MIT,.and I'll get into that a .little bit later /SKPW-RBGSer, .and it's actually been fantastic.for us. . So the booths, and you.can see one on the screen, are .configured to be shipped in a .variety of different ways..We were doing it eight at a time.in these steel caddies. In .reality, we could shift two at a.time all the way up to 16 at a .time, depending on what's needed.in the polling place..That's what we wanted to start .doing, .sending them to polling places .with high .turnout, a big booth, where we .might have 16 booths total. .Some of you may know this as .well, when you send two booths .to a poll worker that's been .there for a long time, they .freak out because they're used .to seeing eight booths, and it's.a big change for them. . District boundaries .are also changing the complexity.of how we're sending out booths..This is really happening now .with the district Palooza, I .like to call it. We're going .from at large districts to .district-based elections in .Orange County. We're working on.15 jurisdictions that are .changing between now and .November, and we have 400 ballot.styles in Orange County that .really change the dynamics of .the number of booths that are .sent out..Some of you in the room may send.multiple ballot styles to a .single polling place. I won't .do that. So if it has a ballot .style, it's going to have one .ballot style within that polling.place. . And you can see there .just on that map we have 400 .different ballot styles on .average throughout the county. . The data preparation .itself, what is the data we're .looking at to make these .logistic al decisions? And .that's what I want to jump into..Projected turnout. So we wanted.to start forecasting turnout by .precinct so that we could have a.better handle on what we might .expect to show up on election .day and what we might have in .terms of the equipment that we .would need on election day..So we first started looking at .registration trends..This is a heat map of Orange .County. I want to just orient .you here for a second..That's Surf City, USA, .Huntington Beach, the coast is .to the left of that. And I put .the happiest place on earth, .which some might argue is this .room .today, but this is the .Disneyland on the West Coast, .Anaheim. So that kind of gives .you an idea of the location. .But when I start to put in -- .this is like a building .thunderstorm. The density of .the voters in Orange .County really are concentrated .in the north central part of the.county, and .those really hot red orange .areas mean there are 5,000 .voters per square mile in Orange.County in those particular .areas. Just to give you some .perspective, we have 1,000 .square miles in Orange County .compared to San Bernardino .County, which I think has 8,000 .or 9,000 square miles. But we .have 3.2 million people packed .into that 1,000 square miles. .We have much better weather than.Los Angeles County and less .traffic. So if you're ever out .that way, come down to Orange .County. . So anyway, when you .look at the density there, this .starts to give us the layer we .use to start forecasting what .the turnout is going to be. In .data forecasting, I think, is .very interesting. I like to use.the weather analogy because, if .you get the next day weather .forecast, it's usually pretty .accurate, 85 percent of the time.it's accurate..If you go ten days out, it's .about 17 .percent accurate because you .don't have as much data inputs .coming in. It's the same thing .with what we're dealing with..The more data inputs we can get,.the more forecast we can .provide. . On that historical .turnout, when we took the .density of those voters and .started looking at ten years .worth .of data, going back ten years .per precinct, we were able to .start predicting some forecast .going from 25 percent turnout .all the way to a 75 percent .turnout. I want to point out .two things to you on this map in.particular. If you drew a .circle around Orange County, .along the coast and up around .the borders of Los Angeles .County, we were predicting very .high turnout in those areas. .The density of voters not as .high along the coast, but the .engagement of voters is much .higher in most areas. So that .was one thing. .The other is right in the middle.of that hotspot, you see a lot .of blue there. That's the city .of Santa Ana, very dense voter .population but not a lot of .engagement in terms of turnout .generally, and that's going to .change here in just a second. . Then we brought in .time studies because this is one.of the things we need to be able.to use the optimization .calculator that Charles has .provided election officials..So we started using every ballot.style, or an average of ballot .styles in every election to .conduct time tests because .that's really important. A .couple of things about time .tests on the ballots. The first.is that you need it for that .optimization calculator, and the.second .is how many votes can be stored .on a VVPAT roll. I don't know .if you use a paper trail here in.Pennsylvania. But in California.we use them. On a California .ballot, you can typically get .about 120 votes on a roll of .paper. That's really important .to know on election day in terms.of being prepared for that. .When you look at the times -- .and I just want to give you a .quick sample here of these .random time studies. This is .from 2016. The sweet spot right.there it is that average, 8.25 .minutes to vote that ballot..That's what we use for that .calculator. Now we go to the .inputs. I'm getting the hook .here. I'm going to go as quick .as I can. . So using these data .inputs, we used the queueing .calculation, and by .the way, this is basic queueing .theory. Charles has distilled .it for elections officials, .which is fantastic, but you can .use -- if you don't use that, .you can use other queueing .calculators. It works well. . So I want to use an .example from Irvine, California..Irvine is one of our largest .cities in Orange County. It's .right there in the middle. And .you can just see that hotspot of.dense voters in one particular .area of Irvine. So that's where.most of our polling places are .located..But the example I want to use is.a small polling place in a very .large precinct, serves a lot of .voters but not a lot of density .in terms of the voters..So it was a good example to give.you. .So what we did in the .calculation, we had 390 voters .that turned out in the 2012 .election..We forecast for 2016 an 80 .percent increase..That may sound significant -- it.is -- but we're not taking into .account the .history of voters that are new .voters /*ers, that are .registered voters since 2012. .We don't have their information..So we don't know the turnout .necessarily. We backed a lot of.that out..But at any rate we ended up with.702 predicted turn out for that .particular area..We then used an eight-hour span .for calculating 84 voters per .hour. Why not 13? Because .that's the length of our .election day. You don't have .busy times on certain points of .the day. We're averaging it out.to an eight-hour day. So 84 .votes per hour for the arrival .time. .And based on that, we plugged in.8.25 minutes for the time to .vote. We established a .15-minute wait time with a 90 .percent service level, meaning .90 percent of the time people .are not waiting more than 15 .minutes. And that gave us a .booth count of 13. If I go over.12 in my jurisdiction, I have to.add a second setup. So we .backed out one booth and sent .them 12. They used to get 8. .How did it work?.I just want to show you real .quickly how it worked. .What we did is we then took this.data in all the 2,700 precincts .I've got in Orange County, and .we did the exact same process .that I just showed you in that .Irvine example, and that gave us.calculations from 4 booths .all the way to 16 booths we .would be shipping throughout the.county..And measuring the outcomes .obviously is important for all .of us. A couple of things to .consider, the number of booths .that you have. That may change .how you're doing it? The .facility size..If you send them 16 booths, .obviously, sometimes they can't .handle 16 booths. We get that. .The number of votes for the .paper trail. That's important. .The bottlenecks for the check-in.stations. This is obviously not.breaking news .for all of you, but that .certainly is where the choke .point is. If you can add more .check-in stations, that really .helps. . In that cube, the .sweet spot is in the middle in .those colors. If we go to the .variations on the outside, we .missed our forecast..So when you look at the map, .this is 2016. Overall, we did .pretty well. When you draw that.circle around the map, our .forecast was pretty much on the .money. We were able to send the.right amount of booths to the .right number of polling places .based on data, not on gut .feeling, based on actual data. .The thing that killed me, .though, was right there in the .middle of Santa Ana. Remember I.told you at the beginning we .were forecasting a low turnout, .we actually had a very high .turnout because they were .energized, irregardless of what .side you're on in the Trump .campaign, they were energized .with that election, and they .turned out in very large .numbers. Caught us a little bit.off guard, but we still were .able to handle it. As a result,.we had reports of 17 polling .places with long lines, out of .1,200 polling places. So it .does work. . One last pitch, and .I'm going to turn it back over. .Because this is a data summit, I.encourage you to visit our .website. We put every amount of.data that is publicly available .on the website, and this is what.you can see in these data rich .tabs that you can click through..We're now doing it in realtime .for .data visualizations as well, .OCvote.com. Thanks very much.. >> Thanks, Neal..You got to Lord it over the L.A..County people. .>> All right. Before I jump .into my presentation, .I'd like for most of you to .start thinking about I'm going .to be .presenting an interesting data .analytic that use what's we call.machine learning. Some of you .are more familiar perhaps with .some of the more hype terms in .this field, like artificial .intelligence. I'm not going to .use that term..But there is an inherent danger .in the analytic that we've .created, and so I'd .like, as I jump into my actual .presentation, you'll see a .number of different slides..Start thinking about what would .be the inherent danger to this .approach? And we'll talk about .it a little bit later as well .because I think it's important .to realize that, while data can .be very, very powerful and a .very useful tool, if use the .inappropriately .or unwittingly, it could .ultimately replicate patterns .that you don't want to .replicate. .Okay. So let me go ahead and .jump into this..So I guess I'll have to point..So ultimately, all of us have to.find enough poll workers to work.on election day. Ultimately, we.need to find enough folks to .staff close to 4,600 precincts .on election day. That usually .amounts to trying to find .close to 26,000, upwards of .30,000 to 33,000 per large .election number of people that .would be willing to show up and .spend about 13 hours of their .time .for a very small stipend that we.give them.. So the inherent .problems with this, as we .recruit people, folks cancel .last minute. The worst .situation for us is when they no.show, no call, no anything. .It's particularly problematic, .and I guess for those of you at .that .administer elections, you know .how problematic it is if it's .the inspector that doesn't show .up on election day with .materials and supplies. So if .that happens where too many .folks don't show up, we might .end up with some long lines, and.we might have folks grumbling .about these types of .issues, which, of course,.leads to other issues such as .low voter turnout, so on and so .forth. So these are all .terrible outcomes that we're all.trying to avoid, of course. . So let me just be .clear. I have received some .criticisms about .this slide. What I mean by .optimal selection is optimal for.us is a physical human being .that is showing up on election .day and wants to work, right?.Irrespective of any demographic,.political leaning, anything to .that effect.. >> Tacky sweater..>> Right. All that .notwithstanding. So we really .do want to find the folks .that are excited to be there .providing wonderful service on .election day to our voters. . So the question for us.is how do we find these people?.Without having to call 35,000, .40,000 different folks..So I started with looking at a .lot of historical data, and I .think it's a similar process to .what Neal was looking at with .historical outcomes of who's .showing up and who's not showing.up..Ultimately, we ended up using, I.think, data going as far back as.2000 to train the algorithm we .had created, and we used the .2014 election as our test data. .And I'll explain a little bit .more about that. Ultimately, .what we were finding is .without using any fancy .algorithm, just simply our .recruiters' intuition, we were .winding up with a 67 percent .success rate for recruitment, .meaning of all the people we .called to try to recruit them, .67 percent were indeed sticking .to their commitment or not .telling us politely or .impolitely no thank you or .whatever else they tell us. . And then that was for .the November. Then for June, we.were at 78 percent. So clearly,.there's a lot of room for .improvement in terms of having .better .recruitment success rates for .our poll workers. .So for us, when we were starting.on this data analytic project, .we were asking ourselves key .questions. One of the core .questions being how do we really.differentiate between those that.are committed and those that are.not committed? In many cases, .our poll workers are looking at .a list of folks that had worked .previously. The natural .intuition is, oh, they worked .last time. They're going to .work this time. That can be .true. It could also not be .true. So the question for us is.are there other data elements .that lend themselves .to predicting the truthiness of .that ultimately?.So we really want to know, of .the intuitions that our poll .workers have developed -- and .mind you, you probably all have .very talented poll worker .recruiters that have gotten to .know their poll workers and the .folks that come and show up and .have a sense as to, oh, if I .call this person, they'll show .up. Oh, this is the wrong .election. I shouldn't call .Fred, right? So there's some .really good intuition. In terms.of data, how do we model those .intuitions from the data we've .collected on poll worker .recruitment?. Importantly, beyond .the intuitions our poll workers .have developed, are there .patterns that could be .mathematically derived when .applying a machine learning type.of algorithm? So ultimately, .the goal of this project -- and .this is the important thing I .would like to leave all of you .with, which is which algorithm .to use or not use, but when .using data, it's important to .have a very clear definition of .what your business problem is. .So what are you trying to solve?.And the clearer and more defined.that .problem is, the better result .you're going to get, the less .ambiguous the data is going to .be, the more accurate .whatever it is that you're the ./* creating is going to wind up .being. . So for us, the .question isn't could we find .enough people to work? Because .ultimately, we were recruiting .from a list of about 120,000, .right? And then beyond that .list, we have about 5.2 million .people that could potentially .serve as poll workers in L.A. .County. So there's a lot of -- .there's a huge pool to recruit .from..But for us, we wanted to .minimize -- so the business .problem for us was minimizing .the negative outcomes that .were occurring on election day, .being .cancellations, no shows, so on, .and so forth. .And the reason why we settled .on that was we /* we felt the .cost of incorrectly a negative .outcome was much greater than .the opposite situation where, if.we were to miss out on certain .poll workers, there's a lot of .other folks we could recruit .from. But if we were to get it .wrong and get the wrong people, .then we could wind up with long .lines, we could wind up with .poll locations not be open, .emergency opening, so on, and so.forth. .So the question for us in terms .of data and data elements that .were available to us is what can.we use? So some of the things .we know about our poll workers .are obviously some demographic .data we collect from the voter .file..We know gender, and we know age..Do we know anything about civic .participation?.Because this is an important .thing in which we would think .would drive participation by a .poll worker is are they civic .minded? Are they part of that .community?.We ultimately realized, of all .the metrics that are out there .or imagined metrics we would .love to have data on, the most .important was voter .participation. Are they an .engaged voter? We found that of.all the data elements we ended .up using, voter participation is.probably the best predictor for .whether or not they're going to .show up and be a poll worker. . Likewise, does .previous poll worker history .predict future poll worker .history? We have all the data. .To some extent it does, it's an .important predictor, but not the.most important predictor. .Likewise, is there a question .about long distance traveling? .Would someone want to travel .from the top of L.A. County to .the bottom of L.A. County? Is .it's a very large county. .Probably not. Likewise, we also.have the specific data that .tells us what was the outcome? .We call it response data. . At the end of all .this, of all the patterns that .existed, we realized there .wasn't just one pattern. I .won't get into how we discovered.additional patterns, but for us,.we realized there were six .different types of poll workers .that existed that could all be .predicted on. We didn't want to.throw everyone into the same mix.and predict the end result. We .wanted to split things up and .get better results, and that .ended up working. .So I won't get too data sciency .here, but I guess the fair .question would be, well, what is.the prediction algorithm doing? .At the end of all this, the end .output is just one data point. .That's what it's doing..It's creating a score between 0 .and 1. It's likelihood of .showing up or not showing up..So in any case of any of these .types of classification .problems, if it's greater than .50 percent, that's the positive..If it's less than 50 percent, .that's the negative. .Now, for us, if you guys .remember, that wasn't the .business problem we were trying .to solve. We were trying to .minimize our negative outcomes..So 50 percent plus one isn't .good enough for us. So as we .developed the different .thresholds, we realized we .wanted to set that 50 percent .much higher, closer to maybe 80 .percent for certain groups. So .not every one of the six groups .that we had created have the .same threshold. It differed .based on the data. But that was.the idea..And by the /SE setting the .threshold higher, as you can .imagine, it means you're being .more discriminating with the .actual algorithm. . So we ended up using .machine learning..I listed two of many that we .used. Anyone that wants to geek.out afterwards, I'm happy to .talk about gradient boosting and.random forest. I won't do that .right now. .Okay. So what were some of the .results? I'm sure you guys were.wondering..Before I get to 2016, 2014, .obviously, we didn't have the .algorithm at the time. We used .it as test data..Using the algorithm, we moved .from a .67 percent success rate to .closer to 86 percent overall .outcome. That's only test data,.though. It wasn't a real .election. It was just simply .giving the algorithm new data it.had never seen for 2014. And .then the general election, we .moved from -- sorry. The .primary election, we moved from .73 percent or 75 percent, to .about 88 percent or 89 percent..So clearly, it's performing the .way we intended it to. And the .no shows and cancellation rates .were also minimized, closer to 1.percent and 2 percent, whereas .before, without the algorithm, .we were right around 10 to 13 .percent no show and .cancellation. And we don't want.to be there. .So moving on to a real election..This is now the algorithm being .used on a live election. For .the June 2016 primary, the .ultimate success rate that we .saw without using the algorithm .was 73.6 percent. The no show .rate was around 13 percent..And the cancellation rate was .around 11.1 percent. Why am I .differentiating? Like any new .technology, adoption is not 100 .percent. We had a few groups .that we had to look at, those .poll workers that didn't want to.use the algorithm and those that.did. In this case, the overall .success rate without the .algorithm is what you see. With.the algorithm, we went from low .70 percent to 85 percent. For .those poll workers that were .actually using the algorithm, .they were seeing a much higher .benefit to their success rates..Lower no show rates of those .folks that they recruited, lower.cancellation rates as well. . Next..Similar data for November 2016. .I won't go ahead and explain all.the As, Bs, and Cs. Those are .just status that we use. What .matters, though, is for the 2016.without the algorithm, we were .at a success rate of 71.7 .percent. No show rates around .6.4 percent. Cancellation rates.around 6.2..With the algorithm, 83 percent, .only 1.9 percent no show rate, .4..9 percent cancellation rate. So.in all cases across multiple .elections, the algorithm .outperforms -- I don't want to .use the term outperforms the .human because I'm not suggesting.we move to robots..But using the tool versus not .using .the tool, those recruiters using.the tool are experiencing .greater successes .in staffing those precincts, and.that's the whole point of what .we were trying to do with this .algorithm. .As, I think, Charles had noted, .I no longer work for the .registrar/recorder doing this .wonderful work. I still work at.L.A. County. I'm doing health .services..I wanted to provide contact info.for .the data scientist I worked with.and trained and is still there .doing the same work. She has .wonderful ideas. She knows .these algorithms as well as I .do. For those who have .immediate concerns or immediate .questions I'm not able to answer.today about this type of use, .that is her contact information .as well.. >> Thanks. Melissa? .So we have a question up here .about which clicker is the one .that's suppose to be working? .This one? Not yours..Well, here, let's try this one.. >> Let's see what .happens..First, before I get started, I .would just like to say, now that.I'm at PEMA, this is like coming.home. I used to work at .Pennsylvania Department of .State, and it's great to see so .many faces here. I'd like to .thank the EAC team and the .Department of State team that .worked so .hard to pull all this together .and for you to be here for .valuing exactly what we're .trying to figure out to do to .help everything kind of improve.. I'm going to be .talking today about emergency .preparedness in elections..I'd like to thank Neal actually .for helping open the doorway to .my presentation with all the .talk of forecasting. .So when PEMA operates within an .emergency management structure, .we have five phases we work in. .It's our prevent, prepare, .respond, recover, and .mitigation. The Department of .State actually works .in very similar ways in .preparing for executing and then.following up with after actions .after an election. So it only .makes sense for all of us to .work together. .We'll activate our CRCC, which .is our Commonwealth Response .Coordination Center, and that is.basically our EOC in the .Commonwealth, and we'll bring in.other departments that I'll talk.a little about later on, but it .allows us to support the mission.of Department of State on .election day, both primary and .general. . Primary threats for .the election day and/or in .general that PEMA .faces when coordinating with the.other agencies enterprise-wide .are flash flooding, extended .closure of limited access or .core routes, hazmat, human .caused events, power outages or .phone outages. Everyone kind of.immediately nowadays is .currently thinking cyber, but .these are the everyday .environments that can cause .flash flooding, downed power .lines. You can have a hazmat .event that would make a polling .place inaccessible..So these are different .environments. Really, I think .I'm the loudest person in the .room, and they still have to .push the microphone closer to .me. Anyhow, thank you. .What we're going to look at .data-wise for today really is .kind of one of the things that's.most important. It's .climatology. So if we look at .the history of Pennsylvania from.the late '50s till present day, .we will be able to know that in .April and May, which are the .months of our primaries and .November for .our general elections, we have a.higher .probability of wind, hail, and .tornadic activity during those .months. So knowing that and .going in, we are going to want .to be able to coordinate with .our emergency management .coordinators in the 67 counties,.just as there are the 67 .election directors and we're in .the process of trying to marry .those groups together and make .sure that they're communicating .the way we are at PEMA with .Department of State to support .that mission on election day. . One of the other .things that's very interesting .is, if we look at .climatology, we can see that, as.the atmosphere builds during the.day, the primary release tends .to really start to happen around.2:00 p.m. and follow through .until just after 8:00 p.m..that's one of the primary voting.windows. You're talking about .rush hour traffic, trying to get.Johnny to soccer practice, and .doing all of those things. You .add in a weather environment, .and that can change a voter's .entire day. It can impact the .polling place in very unique .ways, which we'll look at as we .go forward. . There are a few more .slides in this presentation that.I'm going to kind of go by a .little quickly, but I wanted you.to have the information for .after to be able to look at. .Again, you can see here that .April and .May, we see an uptick in hail .reports as well as SameTime .frame in the evening and the .same with tornadic activities. .And that doesn't necessarily .mean a tornado, but it could .mean the .environment for one to be create.ed..So we also see that it's also .the SameTime of day.. And this is just a .slide identifying since 1950 the.types of tornadoes that we've .had in the Commonwealth and that.are primarily EF-1s. So what .we're looking at here is .actually a slide within a slide..We do a morning briefing at PEMA.with .the enterprise, with the entire .Commonwealth of Pennsylvania and.its partner agencies. Our state.meteorologist is probably one of.the best holes we have in the .Commonwealth, not just for .election day .but in general, and it's really .an example for how government .works. . Our state .meteorologist will put this .together and brief everyone. .This was the forecast on Monday .the .14th, this year, for primary .election day on Tuesday, May .15th..If you look at the center slide,.the center image, I should say, .you see that the storm system is.kind of covering a good portion .of the Commonwealth, but we all .know that weather can change .very dramatically. So now what .was the forecast and what did it.look like on Tuesday on the .morning of the election?.The storm system changed .dramatically. You can see the .time frames of when they were .expected to come through was .going to change. You can see .the coverage area changed as .well. This storm system ended .up going up into New York State,.down through the Maryland line, .and crossed from the west to the.southeast. . So what does that .mean? Let's actually look at .what we were able to pull from .our raw data and go .through GIS and figure out what .this means..So these are real case scenarios.that happened on May 15th. .These are epicenters of all of .our major cities with a five to .ten-mile radius around them. .One of the things they do is .work closely with Department of .State, and just days prior to .the primary or the general .election, they send PEMA an .updated list of all the polling .places within the Commonwealth. .That's this image. This is .every single polling place .within the Commonwealth for the .May 15th primary this year. .The next thing I'm going to be .adding in is our weather events..So the weather intensified .throughout the course of the day.and this storm..Basically, it hit Dauphin County.and .Harrisburg around the 5:30 hour .and continued on a southeasterly.trajectory from the Commonwealth.going from New York to Maryland..You could not avoid this storm. .This follows along the .climatology information we just .talked about. .In Wayne County in the far .northeast, we had an EF-1 .tornado with 105 mile an hour .wins. We also had the same .condition with an .EF-1 tornado at 105 miles an .hour in Cumberland County that .came through. There were .downpours, outages, and we have .to look at and consider what the.continuity plans are for the .election directors and what's .going on in their areas. Do .they have generators? What do .they need for their voting .systems?.How do we support that mission .both at PEMA, and then how do we.work with our .67-county EMCs as well as our .911 coordinators? We reach out .to them a week before or a .little bit more depending on the.weather environment, and we talk.to our 911 coordinators and our .emergency management .coordinators in the counties and.advise them of anything that .could potentially impact a .polling place. . And on election day, .primary or general, we have a .specific contact .in our 24-hour watch within our.CRCC, within the building at .PEMA, that coordinates and .connects directly to the .Department of State to let them .know what's coming through..We also work with the PUC, the .public utility commission, to .find out where the power outages.are so that we can communicate .those as well to Department of .State should it impact any .polling locations and what does .that mean?.So when we talk about PUC and .talk about power outages, this .is just a snapshot of that day..As I mentioned, the storms .intensified .as they went through Dauphin .County and then towards the .east. You can see red bad, .green good..This is PECO and PP&L, and just .a sampling of them. The power .outages were massive. We had .over 16,000 power outages .between the 6:00 p.m. and 8:00 .p.m. hours..You can look in Wayne county we .had 5,300 power outages in that .area. We want to be able to .communicate that information to .Department of State so they can .communicate to the county .election directors and vice .versa with .the EMCs and the 911 .coordinators in their counties. . So what are we doing .next?.How are we -- what are we doing .next as a Commonwealth, as an .entire entity?.Because when we activate our .EOC, when we activate the CRCC .on primary day and election day,.there are multiple agencies .there. Department of veterans .and military .affairs are there, Governor's .office of homeland security, .PUC, public utilities .commission, and we communicate .all of that information. Right .now PEMA, Department of State, .and a number of other agencies .are currently putting our 2018 .November general election plan .together. .The next step is going to be .trying to figure out a way -- .and we've already opened those .lines of communication between .the 67 county emergency .management coordinators and 911 .coordinators with the election .directors because we want at the.county everyone to be talking to.make sure they're talking the .same way that PEMA is talking .with the Department of State to .support their mission on .election day. .Beyond that, Department of State.and PEMA have come together for .some regular meetings to talk .about how .we can improve more in the real .world as well as in data sharing.through face to face meetings, .through an increase in .technology, both before, during,.and after an election. Some of .that is also including the .possibility very early .discussions of creating a .singular dashboard that would .allow all of the stakeholders .and parties to be involved where.we could .upload information from PennDOT,.the turnpike from 511, so you .can look at where there are .traffic incidents and .infrastructure, and they can see.all of that information as well .as outages and other information.that's coming through. .The next thing is be prepared. .What are you doing to be .prepared?.Do you know where all of your.polling location alternate .locations are? Think about .that. When you do have .alternate polling .locomotives locations, do you .have the communication plan to .communicate that .to voters in the precincts so .they realize the voting location.has moved. Those are things you.want to stop and think about. .We're open to having that .conversation..ations, do you have the .communication plan to .communicate that to voters in .the precincts so they realize .the voting location has moved. .Those are things you want to .stop and think about. We're .open to having that .conversation. Department of .State is open to having that .conversation. And we look .forward to discussing that as .you move forward.. >> Mark?. >> Seems like we're .having an issue with the clicker.again.. >> I thought it was .just me. . >> We need to go back..If you can go back to the cover..All right..I am Mark Goins. I am from the .great state of Tennessee. There.truly is a Rocky top. I grew up.20 minutes away from Rocky Top. .In Tennessee, I like to think we.have a positive attitude..So I realized during this .election season that it can be .very stressful..I always relate elections to .cracker jacks. You never know .what the surprise is going to be.that day, but something's coming.at you. . I'd just encourage you.to choose to have a great day..Every morning wake up and say .I'm .going to have a great day. And .I also encourage you to choose .to learn something today..And being from Tennessee, we are.very family friendly in that I .always start the presentation .off with introducing my .wife and letting individuals .know who my wife is. So there's.my wife. I'm Dolly Parton's .trophy husband. You always want.to know what he looks like, and .I'm him. .No, we're very proud of Dolly. .I'm not married to her. I have .a lovely wife Rebecca. Anyway, .Tennessee is certainly known for.Dolly Parton and other things. .I got invited to do this panel, .essentially I was told the topic.would be about tabletop, to go .into a tabletop discussion. And.part of the reason was we .recently .did a tabletop with all of our .election officials in Tennessee .on the county level, essentially.400 individuals, and it's a .photo of 400 individuals in the .room doing a tabletop exercise..And I was very blessed in that .we were .able to get a lot of ideas from .the Harvard project at DP3. And.they gave us some tools to .conduct a tabletop. .Now, in all candor, the tabletop.that we did didn't have -- it .wasn't all data, but I'm going .to go through in just a moment .on how it could be all data. It.certainly had a data component .to it. But essentially, if .you're not familiar with an .election tabletop, the .way that ours worked was during .phase 1 you would study for an .election. You would plan for an.election. We literally gave .everyone a notebook. They had .to put their plans. We'd divide.them up into teams of ten, and .they to come up with a plan on .how to prepare for an election, .and then on election day, they .had to execute that plan. . And part of that, .though, we were going to hit .them with various, what we call .injects. For example, in our .inject, we hit them with a .tornado. We wanted to know if .they planned beforehand, and if .they didn't plan that .in phase 1, then in phase 2, .they were in trouble. .Basically, for the tabletop, .that's the worse we could .imagine..Anything that could go wrong .happened. .What we did incorporate in phase.1, we did give them a budget. .Also during that period, we told.them turnout is going to be .high. So we told them that .information. We also told them .during that time that 21 states .had been scanned by a foreign .actor. We gave them a choice. .They could reduce polling .locations, early voting sites. .I'll go into early voting in .Tennessee in just a moment .because I realize Pennsylvania .is a little different than .Tennessee in this aspect. I .know there's folks watching .across the nation. Anyway, they.had a choice..They could save money by .introducing voting sites. The .problem with that would be later.on they're going to have long .lines, and .they could take that money and .put it to cyber security, or .they could do just the opposite .and not spend any money for .cyber security, and guess what? .On election day, they're going .to have a cyber activity. So .data was involved. .But overall, as we go through .this -- as we're looking .through, and I may do a data .tabletop..In doing a data tabletop, some .of the relevant things we have .to throw at them, we'll all deal.with data, and .we'll talk about that in just a .moment. But I want to make this.statement, and maybe this is one.thing you learn from our .presentation, and I truly .believe this. As on election .official, if you do not use .data, you do not know how to .plan an .election, and I believe this. .As we go through, what data is .important? We throw them a .number of relevant data and .irrelevant data..You have a finite number of poll.books and machines. But in .Tennessee, we have an early .voting period that starts 20 .days before the election, and in.general, with very few .exceptions, will end 5 days .before the election. So .sometimes that relevant data at .the beginning of the process can.change as you go through it. .Let me give you an example. If .we were doing a tabletop .exercise, polling location .number 1, we might say they're .having a 60 percent turnout .during early voting. You have a.finite number of voters, but 60 .percent of those are turning .out. Polling location number 2,.turnout is extremely low. In .the planning phase as an .election official, now you have .that data, but you also need .more information. Or at least I.would take more information. .What I would do is go look and .say was .there something on that ballot .in precinct number 1 that's .causing that turnout or a .candidate there with a strong .get out the vote effort? It may.be you're going to run out of .voters in precinct number 1 and .you can ship resources to .precinct number 2..Or it may be you have a weak .ballot in precinct number 2, and.you need to ship the resources .you have, poll officials, .whatever the case may be, to .polling location number 1. .You also have a finite amount of.money. Money is always going to.be an issue in an election. .There's so many things we can do.that would be great, but you .also so take that in context .with money..Vendors can be an excellent .trouble shoot as far as .providing data. . For example, we had an.issue with pollbooks, where .unfortunately the .wrong early the /* voting data .had been entered, so on election.day it appeared individuals had .voted who had not. Now we .require vendors to put that data.on the opening page of the .pollbook. If I have a pollbook,.I know how many of my voters .have voted early. I also know .from the election commission, .they've sent a data showing this.is how many people voted early. .If those don't match up, we know.we have an issue, and we test .that before we send it out to .the polling locations. .With the ballot, sometimes you .have to create your data. Now, .if we were setting up a .tabletop, the only thing we .would give them would be a very .long ballot, and in Tennessee .every eight years, that's .exactly what we have. Every .judicial race is on the ballot. .It's a retention type election..And so it's very long. So how .do you create that data? Well, .we wouldn't tell them, but my .hope would be, when they were .doing their planning, they would.do a mock election. They would .take members from across society.and figure out exactly how long .it's going to take to vote that .ballot..In doing so, then that can help .you plan. .The next thing is looking at .relevant data. Again, we have a.U.S. senate seat in Tennessee, .six years ago it wasn't .contested. It was contested, .but pretty much nominal .competition in Tennessee for .that. We haven't had an open .senate seat in a decade. So the.tendency is to look at six years.ago and see what turnout would .be. Frankly, I think that's .wrong..I think you've got to look back .to 12 years ago, to where we .actually had a contested race, .money was being spent, and I .truly believe that the candidate.spending money, there's a .correlation with turnout. That .helps get the vote out, or at .least it does in Tennessee. So .I think that's important. It's .important to know what happened .six years ago, but it's more .important to know what happened .12 years ago. .As Neal said, you've also got to.-- so now you have that data. .You have that information. But .you've also got to look at .what's the voter registration .trends right there?.And you've got to adapt that .data accordingly..Number of voters, that's .obviously how you plan -- we had.a new administrator come in .several years ago. By way of .example, they had ten precincts .and 30 machines, and I'm .simplifying this. In their .mind, they sent three voting .machines to each precinct. .Well, that's not how it works .because some of your precincts .are going to have more voters, .other precincts are going to .have less. Like I say, .sometimes that ballot may be a .little different in the .different jurisdictions that you.have. .Age of voters. I have some .administrators -- I guarantee .this would happen, and it's a .great thing that I have this. .I'm blessed with this..So we have 95 countys, and some .of our administrators analyze .everything..So if we were doing a tabletop .exercise, we have some .communities that are senior .living centers or senior living .areas like Fairfield Glade. .It's a very popular area in .Tennessee. A lot of seniors .live there. When I hit 40, I .can't read a ballot without .glasses. I need something to .magnify those. So we saw that .was an issue in polling .locations..So we sent glasses out to the .polling locations. Now, if I .have a polling location that's .located close to the University .of Tennessee, I look at the .data. It may only be college .students voting there..I don't have to send as many .glasses. .But in Fairfield Glade, I'm .probably going to need to send .more glasses. Now, I'll tell .you, if you choose to do that --.and this is serious -- if you .choose to do that as a state or .if you want to provide those .glasses, find the ugliest .glasses you can find because .those glasses will walk away..We're getting timed up here. I .see I need to speed along..But absentee requests can tell .you what kind of turnout you're .going to have. . Trends are very .important. I've learned that .people are creatures of habit. .I saw this with my family. Are .we going to church? We sat on .the opposite side. We typically.sit on the right, and we sat on .the opposite side. My son .looked at me and said, hey, .we're sitting on the wrong side..Voters, at least in Tennessee, .tend to be creatures of habit. .If you look at this, the blue .line is the presidential .preference primary..Normally for early voting, 30 .percent. 60 percent is almost .dead on. 60 percent four years .before that, and four years .before that, 60 percent. So we .know when planning elections, .early voting for a presidential .election, we're going to have 60.percent..We need to bulk up early voting .sites. For presidential .preference primary, look how .consistent it is, 30 percent. .It's just the opposite. So then.you need to bulk up on election .day..You don't need to be bulking up .during early voting. .Well, I have that data, so I .also like to get behind that .data. Why is it only 30 .percent? Why does that buck the.trend? For a presidential .preference primary, we're not .New Hampshire. We're not Iowa. .We're later on in the process. .So what happens is candidates .start dropping out. We start .early voting 20 days before the .election..By the time early voting is .continued, half the candidates .are gone..Voters want their vote to count,.so they wait until election day .to vote. . Here's where this will.catch some individuals, and .we've seen it happen in .Tennessee. Although I went over.this in one of our training .exercises, always compare .elections to elections. .Presidential, don't go two years.before because it's different. .Go four years before..And we had.an individual that looked at the.60 .percent as opposed to 30 percent.when preparing for a .presidential preference primary..They thought, oh, this is 60 .percent, but it's really 30 .percent. They thought they were.going to have a very low .turnout. By the time it came to.election day, they were caught .off guard. You've got to know .exactly what data is relevant. . Sir Francis Bacon is .quoted with saying knowledge is .power. I'm going to give him an.amendment. I'm going to give .him an assist today..I think that data is knowledge .and knowledge is power..>> Thanks, Mark. So while the .audience gets their questions .together, I thought what I'd .do is I'd go down the row and .ask each of the panelists a .question that's pegged to their .presentation. Actually, I'm .going to start, although Mark .just presented, I'm going to .start with him because I think .the question .I'll give to Neal, a couple of .you all might want to weigh in. . I'll actually preface .my question to Mark, with .respect to ugly glasses, you .must not have a whole lot of .hipsters in Tennessee. I would .imagine that, in some places, .the uglier the glasses, the more.likely they would be stolen. .The question I want to ask was .just about Mark. So one of the .things you learn when you study .-- get to know election .administrators well is that .election administrators are .natural emergency planners, and .if they're not, they're not .election administrators for very.long. That's been known for a .long, long time..The defending digital democracy .program at the Belfer center at .Harvard, .which you mentioned, has .introduced the election .administration world to the .planning of tabletop exercises. .I'm wondering if you could .reflect on how the tabletop .exercises differ from what you .see as traditional emergency .planning and what sorts of .information .or experiences that they give to.election administrators that .maybe more traditional ways of .emergency planning have lacked .in the past.. >> I think the .difference between the tabletop .exercise that's we did is the .emergencies happened. So now .how do you react?.The typical way we were .preparing was we'd prepare a .plan and didn't execute the .plan. During the tabletop .exercise, it forces you to put .that plan in action. And if you.don't have a plan or you didn't .think of something in that plan,.then you need to go back and .adapt that plan..>> Great..Melissa, I imagine your role is .full of tabletop exercises, so .you might have some thoughts .about that. One of the things I.was wondering .about as you were talking was .thinking about the old Sesame .Street game, which of these is .not like the others? Or maybe .the opposite, which is to .what degree is planning around .an .election., how is that pre-planning for .things that might come up? Any .other large events in the state .that are like that?.What are the similar other .events that folks can get their .heads around..And in your world, how is it .different planning for an .election more difficult than .planning for another large scale.event that might be in the .public's mind..>> Just in case. I'll need to .tell my father they had to do .that. We'll exercise for a .number of different things, both.in tabletop and .then actually by activating the .CRCC and having our agency .representatives come in. It .will be for power plant .exercises and things along those.lines. It will also be for .weather events..We always do a spring weather .and a winter weather exercise .just prior to that season. .But really when it comes down .to preparing for responding to .an election, we do what we do .best, which is coordinate. PEMA.doesn't save lives, but we .coordinate. So we make sure .that all the agencies .and the resources are talking to.one another in that we're kind .of a conduit, if you will, for .that information to be able to .flow. . We've sat down .together and worked some smaller.scale election scenarios prior .to election day, and we're .hoping to kind of be able to .build .onto that, and the next step of .that is actually bringing more .of the county .emergency management .coordinators and .911 and election directors to .kind of practice that. Does .that answer your question?. >> It does. Ben, I'd .like you to talk more about a .topic that you touched on, and .that has .to do with one could derisively .call the demographic profiling .dangers of the sorts of exercise.that you're dealing with.. So when you're .building a model of a sort that .you talked about, how do you .ensure that the down side of .that sort of modeling doesn't .infect the types of decisions .that get made? . >> Yeah, there's a .wonderful book out there, and .I'd highly recommend any .election administrator who wants.to be data driven to look at it..It's called weapons of math .destruction..It's a nice critique of the use .of big .data and these types of modeling.and .analytics. What happens -- so .just understand these types of .analytics aren't creating .patterns. They're discovering .patterns for you. Ask .yourselves a question, who .created the patterns? It's your.human employees. So in the case.of poll workers, it's your .recruiters that are using their .own -- I use the word intuition..Another word that could be used .is bias. It could be explicit. .Hopefully not. But it could .very likely be implicit bias. . So that individual .poll worker are making .decisions, human decisions .that are trying to yield the .best outcomes that they can .yield..But over time and aggregated .together, there are going to be .patterns that exist that could .lend themselves to what I think .Charles is talking about. So .out of curiosity, when you look .at .your poll worker, anyone look at.the age demographic of the .likely poll worker? Yeah, .older. .Now, the question ultimately .is, without passing any .judgment, still a question .though, is that a good thing or .a bad thing? In terms of .planning for future elections, .well, if all your workers are at.the cusp of retirement or .already retired, that may not be.a good thing. You may not have .those -- God forbid, you may not.have a large number of those .poll workers for the next .election..So for logistic planning, maybe .there needs to be some effort .spent on recruiting other .demographic, right? .But, again, without getting into.making the judgment calls, do we.go after this demographic or .not? And that's probably not .that good of an idea to choose a.certain demographic, .but it's important to use these .types of algorithms to uncover .whether there's been a pattern .in things that have been done .before. That's what this .algorithm is doing, it's .uncovering the patterns. If you.could dig into the data using .these algorithms, you could .start realizing and potentially .reversing some of the previous .behaviors that your employees .have used to recruit certain .individuals. .So this addressed that first .question I brought up in terms .of the inherent danger to this. .If these -- if using advanced .data .analytics is used purely blindly.to say, .hey, let's just get high .performance, then you as an .election administrator are at a .risk of just reproducing the .same patterns but optimized by .this algorithm that you may not .want..So the way to combat some of .those things is to dig into the .data and ask yourselves are .these the patterns we want to .continue? If they're not, .great, you just used a tool that.uncovered patterns that may have.been done unwittingly. And then.you could start addressing .the way you go about conducting,.whether .it's poll worker selection.precinct selection. Anything .you have, if there's patterns .and you want to know what the .patterns are and make an .adjustment as an election .administrator, those tools will .be available to help you make .better decisions in that way..>> And then Neal, I wanted to .ask you about just kind of the .resources needed to become the .next Neal Kelley..Maybe I'll make a comment and .then pose the question another .way..I think Orange County really is .an exemplar in terms of using .data, not just using data, but .communicating effectively to the.public..And doing so requires a set of .skills. And so -- and not .everyone is the fifth largest .jurisdiction in America..So if you are like the .thousandth or the 500th largest .jurisdiction in America or the .6,000th largest jurisdiction and.you wanted to play in .this game, where does one get .the types of tools and expertise.in order to do the sorts of .things that y'all have been able.to do maybe at a smaller scale? .But nonetheless begin to play in.this game.. >> Thank you, Charles..I don't know if that is a slight.or a compliment. I'll take it .as a compliment. You know, .that's a great question, and I .get it all the time. . The one thing I always.tell people is what we're doing .is scaleable..So there's a lot of tools that .are .available online that can be .ought liesed for the same types .of data collection and data .analysis we're doing very .simply. And I think that .smaller jurisdictions .that may not have the funding or.the .resources can do at least half .of what .we're doing with online .tutorials and online tools. .There's a lot of that out there..It's very useful right now. .I also encourage folks to .utilize -- and this is a great .place to say this, a community .college, because there's some .great classes on operations .management that dive right into .this very specific topic and how.to handle logistics and data..If you took a weekend course or .even a .six-month course, that's really .helpful .because it talks about queueing .and line management and .logistics, which is very useful.. But I do want to .emphasize, Charles, because I'm .glad you asked the question, .it's not just about money and .about resources..For instance, on the social .media .side, all of that is done .probably in a negative way..My family certainly says that, .by myself. Because I'm able to .send that communication out and .I can do that. If I can do it .by myself, then do it..We don't have a team like Los .Angeles County..>> I have some other questions .I'd like to ask, but I think .we'd like to turn it over to the.audience. It looks like we have.some microphones around, and I .will try to call on people as .they raise their hands. So what.questions do you all have?. >> Hi, I'm Kara Rahn .from .Chester County, an election .official from Pennsylvania. .Just the question to California,.just so we can understand from a.poll worker perspective, what .are the laws or rules in .California or restrictions you .may have on who can and can't be.a poll worker? And then also .who has the authority to fill .those roles?. >> Thank you for the .question. In California you .don't have to be a U.S. citizen .to be a poll worker. You can be.a legal permanent resident. In .Orange County, this is an .example, I support six languages.in the county, and that becomes .very handy because we can't .always find poll workers that .are U.S. citizens that are .willing to come and do the .bilingual support. . So we have the LPR as .an alternative. We also have .the student poll worker program..You have to be 16 with a 2..5 GPA with the permission of .your school and your parents. .We use about 2,000 of those in .Orange County. Ben touched on .it a little. Our population is .aging. Our average age was 67. .When I first started, it was 72,.and that's dropping because .we're putting more young folks .into the mix. But I find it's a.good balance to have the older .folks and the younger folks in a.polling place as opposed to all .young. That's really helpful. .Hopefully, that addresses.. >> Microphone .microphone..>> No, they're not. Let me .touch on that quick. Obviously,.in some jurisdictions, you have .to have a mix between .Republicans and dems. We don't .have those restrictions. It's .all open recruitment..And we allow our inspectors to .recruit their own individual .teams if they want to do that as.well..I have to recruit 10,000 poll .workers for each county-wide .election. Our team has to do .that. So it's a very daunting .task. So we rely on some of .them to help with that .recruitment..>> Just as a followup, I think .that's what makes using this .type of data so much easier .because we don't have those .restrictions. We can literally .go after anyone. For us, the .limitation of the algorithm we .use is using voting history. So.we generally are targeting folks.that are L.A. County voters, but.we do have plenty of poll .workers that are from Orange .County that commute because we .also have the county poll worker.program, which you guys might .have as well. And we have .Orange County residents that .work in L.A. County..So we borrow from L.A. County, .other departments to staff the .polls. . Ultimately at the end .of the .day, it's nice not to have any .of those types of issues to stop.an algorithm from trying to .optimize. You keep adding in .more restrictions, the question .is how much optimization can you.really squeeze out from what .you're already doing? .>> I'll just make a comment as .somebody who's not around here .but has done a couple of things .around Pennsylvania around .elections. It's interesting .because ultimately there's the .question about do you elect your.poll workers?.I've taken that to be an issue .here in Pennsylvania. .There's a question here far to .my right.. >> Yes. Good morning..This is Suzanne Herb from .disability rights Pennsylvania. .One of the things that I think .is .important to remember in terms .of.data that doesn't always come up.is .voters who have disabilities.. In terms of the discussion .that has .just taken place, what efforts .-- can .you describe what safeguards are.in .place to ensure.the ability for people with all .abilities to vote.independently and have a .verifiable paper ballot and also.have a private vote are in .place. .One of the problems that seems .to come up very often in terms .of polling place accessibility .has not just only to do with .polling place .accessibility itself but also .the .ability of poll workers to use .the machines and set the .machines .up so that people who have .disabilities .can cast independent and private.ballots. I know that that has .been a concern of mine on .various occasions when I voted, .and that affects not only my .vote but also lines..If the machines don't work .properly, .it affects lines, it affects .everything. Could someone .please address that point. .Thank you..>> I'm sure we have several of .our panelists who would like to .hop in..Start with Neal. Then I'm going.to call Mark..>> Thank you for your question. .I'm glad to hear that disability.rights Pennsylvania is here. I .do a lot of work with disability.rights California in Orange .County, very close work. One of.the things that I have struggled.with and that I would like to be.able to get my hands on is data .on .the number of voters with .disabilities, and that's a very .difficult data point to try and .capture..We've had a lot of discussions .with disability rights .California regarding that. One .of the things that we do, .though, to ensure that voters .with disabilities .can cast that -- their ballot .privately and independently with.a paper trail is we offer all of.those things in Orange County so.that each polling place is set .up with a disabled access unit .or more depending on the voter .density that allows voters with .disabilities to cast .their ballot with all of the .tools available on that unit .with a paper backup trail that .they can verify. .We also provide ballot calls so .that, if a voter cannot exit a .vehicle or they're outside, they.can push a doorbell, and we will.bring the voting booth to them .so they can vote in their .vehicle. Again, I want to go .back to the data because this .being a data summit, if you have.information or ways to capture .that .data, I would love to talk with .you because I think that's .really important .for us to be able to focus more .resources in that area..>> Mark, if you have thoughts .about that..>> Yeah, Neal is correct..Each polling location -- so .we're a little different system .in Tennessee where we still .primarily rely on DREs. We .haven't truly gone to the paper .ballots, but a few counties .have. Even the paper ballot .counties, we make sure there's .an accessible unit that's there..One thing that's a challenge is .to figure out which numbers are .utilizing those systems. One of.the challenges is we .self-create. We don't want just.one person using that machine..So we will -- even though it's a.machine that is typically for .those that need that .accommodation, we try to get .other voters to go there as well.because we're always concerned .about protecting how that .individual voted. If you have .one unit and only one .voter is voting on that, that .could .create problems of it itself, .plus we don't want to single .anyone out. We have a very good.relationship with two .organization that's focus on .disabilities, someone typically .at our state election commission.meetings there. We also have .bell calls and also do our best .to supply the precincts with .various things. .One thing we did do. We worked .with them and created some .training in a handbook or poll .officials..We did that with them working .from their point of view. . >> A check of my .previous bosses in L.A..County, the VSAP project, which .is the voting systems for all .people. I know that at its .core, in terms of working with .disability rights groups, much .of the work that is done in .actually designing the machines .and designing the experience was.very much tailored to addressing.many of those concerns that .those community advocates had in.being able to provide that type .of good experience at the polls..Obviously, it's a different .problem when we're talking about.vote by mail, and there are .different necessary solutions to.think about that. But for the .data as well, we have looked at .different data sets, and it's .imperfect out there. One can .look at census bureau data, even.down to the census track, even .group block level, try to .massage that up against the .citizen voting age population .data and try to get a better .understanding of your voters in .these areas, which ones have the.greatest need. .So there's possibility for that,.but it's, again, very imperfect..Census bureau data is not .presented to .the public or even to election .administrators at the individual.level. So we can't know all .those things we'd like to know .about our voters. But there are.things that are out there that .you can use to join data .together and get some .approximation..So those are things we've begun .doing as well. .>> Just want to add I'm actually.a little ashamed to admit this. .We had not been doing this .before 2018 and to bring in .disability rights of California .and other groups into our .poll worker training to have .them speak to the poll workers. .It's actually the poll workers .doing the poll training, to talk.about sensitivities and how to .work with voters with .disabilities. It's actually .been great..I'm just ashame we haven't been .doing it for years before. .>> Actually, while we're .waiting. The microphone is .there. I'm going to prime my .panelists..I want them at the end to list .very quickly one resource that .you would like to turn this .group onto that hasn't been .mentioned yet but you just think.it's great that people have know.about. . >> I'm a reporter from.republica, and we cover .elections and voting. We are .undertaking round 2 of election .land this year, which is a .project that aims to train local.reporters on how to report on .elections themselves responsibly.and accurately during early .voting and on election day. One.thing that I would really like .our reporters to do is sort of .establish a relationship with .their local election .administrators..I'm just wondering if you have .any .tips for maybe things that .reporters .miss or things that they can .start to do .to prepare now to cover .elections in your area or across.the country better or more .responsibly than they usually do.on election day.. >> Who wants to bite .first? Just go down the line? .Let's start with Mark..>> Could you summarize that .question again. I'm sorry..I'm not fully understanding your.question..>> She speaks with a foreign .accent. .[Laughter ] . >> What we're trying .to do is sort of help reporters .understand the resources they .have available to them. So that.when a voter comes to them and .says I have this really big .problem at my polling place, the.machine was broken or vote .flipping, they know how to .respond to that and they don't .just report on whatever. They .have understood from the voter..So we're wondering if you can .give us some tips about can .reporters come to your polling .place now and take a look at the.machines? Can they get better .information from you on how -- .like what mistakes you expect to.see on election day that are .pretty normal and your poll .workers are trained to deal with.so they can give that .information back to voters in a .responsible way on election day.. >> That is a good .point on the way we messed up on.our tabletop exercise. After we.did our tabletop exercise, a .reporter found out about it and .said, why didn't you invite us? .Actually, I think that would .have been a good opportunity on .their behalf to instill voter .confidence because all they're .hearing in the news are negative.things about the election .process currently. .But in Tennessee, it's a little .different maybe than in some .other states, or maybe it's the .same. For our process, when the.voting machines are being .tested, anyone can come in and .view that. It's an open process.for the machines to be tested. .So reportered are certainly .welcome. Notice is typically .given to the local media that .that's happening. So that's .always something that can .happen. .On election night, we always .want the reporters to come -- .sometimes reporters will reach .out to us and say they'd like to.come to the state and see what .the results look like as they're.coming in. For us, it really .doesn't help them because we're .looking at a computer terminal .where someone is entering it in .a portal..So it may be better suited to go.to the local commission office .where you can view those things.. But the main thing is,.I guess, from a media .standpoint, what can help the .election officials, it's amazing.what the voter perceives, and .many /* sometimes it's driven .from what they're hearing, and .maybe because it's politically .charged what's going on in the .election. Before a headline is .given or written, it would be .awesome if a reporter would .reach out to someone in .elections to get the know. It .may be that election official .doesn't instantly know what's .going on, so if we could have a .little period to research it and.come back. .In my experience, that's the .things that's kind of hurt us a .little bit is the headline would.say one thing and it's totally .wrong. Sometimes it's because .they've called an election .official and the media has to .get that information out there, .and the election official gives .an answer, and they haven't .researched it on their end. . >> If reporters want .to know about covering kind of .the emergency .prepared ness aspect in .Pennsylvania and other places, .what should they be doing?. >> As far as the .emergency preparedness aspect of.things, we'd suggest that the 12.million-plus population of .Pennsylvania do this on a .regular basis anyhow. But .social immediate why is so .crucial right now, and we want .to find something out in what's .happening right now. Somebody's.tweeting whether or not it's .pouring down rain. So you're .going to see that..So find your social media .resources, follow us at PEMA on .Facebook, on .Twitter, the PEMA director has a.handle as well that you can do .that at. . To kind of go back a .little towards my days at the .Department of .State, if I may be so bold to .jump in on that one a little .bit..Really have as a reporter a .comprehension of election 101. .Your county election offices, as.well as the Department of State,.have spent a .lot of time with prely talented .people to explain that process, .how everything works, allow .e-mail, phone calls. You can .come in and ask those kinds of .questions and find out exactly .how it works in the Commonwealth.of Pennsylvania and what those .things mean, guidances, other .things along those lines. .And you can find a lot of that .on the county and the state's .website..So if I could suggest that, .that's a great possibility. . >> Tips to reporters?. >> So I think to echo .some of what Mark had said, I .would love it if reporters, .especially reporting elections, .were just much more data savvy. .And I say that in the sense of .most of .the data that we use is public .information, and most can have .accessibility to it. But .sometimes some folks don't .understand what data in one .table or one source means one .thing, but in a different area .or a different -- if .you're /HRAOGing at something .looking at something else, it .means a whole other thing as .well. I would imagine, at least.with L.A. County, there's always.a group looking at this data. .If you can make that connection .in the event that you have .questions about what does this .actually mean? I'm looking at .this data that's posted on your .website. It looks odd. Or .turnout doesn't look right. .Sometimes there's errors the .county elections office can .make. And those are all useful .things to bring to the attention.of the administrator. .But it could be very .frustrating, I would imagine, at.least .from when I look at things, when.certain things are being .reported and the context is .often completely .mischaracterized or the data .doesn't actually support the .conclusions that .are being presented, which then .lead to misperception by the .voters and so on and so forth.. >> I have a very .specific example for you..Ask your reporters, the .reporters you know to build a .relationship with the election .official, and then ask them if .they can ride along on election .day for the emergency response .teams..That has been really helpful on .our side. David Whiting is an .Orange County Register .columnist. He recently did that.with me. Just Google that. Or .even our regular response teams .because they see what we have to.deal with on election day, and .it provides a new perspective .for them..>> Great..So as a way of wrapping up, the .lightning round. One resource .really quickly down the table, .and then we're standing between .these people and lunch. Neal, .one resource..>> Real quick, hoot suite, .social media aggregator. Get .all of your social media tools .in one location. It helps .collectsing data, what's .happening on election day and .pre-election day.. >> Datacamp.com. And .I say this -- most of us in this.room who run an election do not .have the budget to go out and .hire a data scientist. I think .that's for real..But a site like datacamp.com, in.the hands of staff that already .have interest in doing the type .of data .analytics that you'd want to do,.that will upskill those folks in.a very, very fast way..And I make the suggestion to do .this .because.the Democratization of data .science is happening very fast. .There are lots of citizen data .scientists that are out there .looking at .the public data we all have, and.they're looking at the patterns,.and if we're not analyzing .what's going on in terms of .services we're providing, .someone else is doing that. It .would be better for you to .uncover those things than .someone else who has a very .critical eye to what's being .done.. >> I know we're .talking data, but I would say .people. Make sure one of the .things you're finding, you're .connecting with folks, both at .the state level, PEMA and .Department of State, as well as .the county level because they .can provide you with the .information, the statistics, the.material, the data to be able to.be responsive on election day .should an emergency happen .within your county. First your .people, and then they can .provide you with the data to be .able to .do what you need to do in time .of an emergency..>> Second thing I read every .morning is electionline.org. .And I think you can go through .there .and see what's going on in other.states, but also like today .there was a helpful article .about looking at maps and using .various things. So .electionline.org..>> Right. Thanks to the panel .and thanks to the audience..Well, we should give these guys .an applause. .[APPLAUSE]. >> So what now? One .hour for lunch. You're on your .own. Come back in an hour. .It's going to be even better.. [BREAK] ... >> Hello, everyone..We're going to start in about .two minutes.. All right. We're .about to start..I want to thank you all for .coming back. Good afternoon..I hope you had a nice lunch and .are .ready to get back into .elections. .I equate this panel similar to .-- and I hate saying this .because I'm in .Philadelphia -- but preparing .for the .Super Bowl as in during the .preseason, you're not thinking, .hey, I'm going to be at the .Super Bowl. You're thinking how.do I prepare for the Super Bowl?.So election day is election .officials Super Bowl..So this is what I equate this .panel to. .So I'm pleased to serve as .moderator for our third panel, .which is focus on data related .to the election day itself. .Data savvy election officials .across the country are .leveraging a number of tools to .strengthen specific aspects of .voting operations, tools such as.mail-in ballot tracking systems,.electronic pollbooks, line .management tools are not .only improving the voting .experiences, but they're also .generating invaluable data that .can be used to inform .decision-making in realtime in .voting and post-election period..Civic groups, political parties,.and candidates who work to .register voters and mobilize .them on election day are also .constantly innovating and honing.their data driven strategies. . The tools used by .these election stakeholders rely.heavily on voter registration .and turnout data generated by .election officials..We'll focus these and related .topics during our panel today..Three of our expert panelists .who today currently serve as .local election officials in .Colorado, Indiana, and Virginia..We're also pleased to have with .us a representative from the .data driven civic group here in .Pennsylvania. Each of them .brings a wealth of knowledge and.experience from across the .country to our discussion today .about .how data is used to improve our .elections and support voter .participation. . On my far right, Amber.McReynolds is the director of .elections for the city of .Denver, Colorado..She is /* she has administered .elections in Colorado for 12 .years and has worked in public .policy and administration area .for over 16 years..Denver's elections commission .has earned a number of .recognitions under Amber's .leadership, including the ballot.trace program, which she will be.speaking about today. .Ray Murphy, who I don't know. .So hello, nice to meet you..Is a deputy director at .Pennsylvania, where he monitors .election key security .and monitors campaigns and helps.the 501c3 civic engagement .programs before coming to .Pennsylvania vote..He served as the director of .grant .making at graham rose's .community fund. .To my left is Michelle White, .the general registrar for Prince.County, Virginia..Including 11 years as director .at Prince William counties. .And last but not least is Laura .Herzog. As the election .supervisor for Hendricks County,.Indiana, where she has served .for 24 years..Laura is also a member of .Governor .Eric /HOL comb's executive .council on cyber security .chaired by Connie Lawson. Each .of the panelists will give .prepared remarks, and then we'll.shift into question and answer .period. And again during the .question and .answer period for you, we ask .that you .identify yourself, speak into .the Mike, and most importantly .what? There we go. There we .go. We'll start with Amber.. >> Okay. Well, it's .great to be here..I was participated in one of the.election assistance commission's.data summits a few years ago, .and I find these to be extremely.valuable to talk about very .critical issues facing election .officials and really have good .discussions and talks about how .to best improve that. .So in Denver, this is my 13th .year in Denver, seventh as .director of elections..I came to this with background .actually in data analytics and .what have you during my graduate.program. So the analysis of .data, what it can do in terms of.policy making, continuous .improvement, process .improvement, all of .that is something that I had a .great understanding of coming to.the job. And when I started in .Denver, there was very little of.anything with regards .to data management, innovation, .data analytics, in any way..So we've spent a great deal of .time building that up, so much .so that we use it to improve our.processes..We've used it to advocate for .significant changes in .Colorado's law..In fact, we were sort of the .driving county, and I helped .write a lot of the .legislation that we passed in .2013 that provided for ballot .delivery, vote centers, same day.registration and sort .of all the innovative policy .reforms we all have. .So data is powerful, and that I .know very, very well from my .experience in Denver but also .prior to that. .So I put this up.electionopoly, and you see on .the .slide, gerrymander boardwalk, .community chest with campaign .finance. But the point of it .that I always try .to illustrate is with elections .there's tremendous games that .can get played .with processes or policies or .even just inconsistencies and .difficulties that often occur .for voters. So we've really .tried to make the rules of .engagement clear, advocate for .policies that make more sense .for voters, and that better .serve them in a more effective .way. .One of the other things, I've .got a couple quotes in the slide.deck about data. This one I .like. It's Jennifer. She is .the founder and executive .director of Code For America, .and our ability to do great .things with data will truly make.a difference in our lives and in.our world. Sorry, I think some .of that is cut off there. It's .a little formatting thing. .So given that and kind of .changing the world and making .things better in this .environment, which is the .eaction.-- elections world, for me, this.started at a young age. There's.many things going on in this .picture, but the one that sort .of makes me laugh and my parents.laugh at this picture every time.they see it is that I .was probably strolling around .thinking about doing a couple .different things in this photo .but then also thinking about .whatever I was going to do the .next day or whatever I was going.to do beyond that and what have .you. So that's kind of -- I .think it's sort of indicative .how I see stuff in elections, .trying to always look for .improvements, ways to get .better, and what have you. . Our approach -- and .I'm going to talk about ballot .trace and sort of how it .supports our ballot delivery .model. But the reason we .developed ballot trace and why .this matters so much is we now .in Colorado have a ballot .delivery system..We had permanent vote by mail at.great, great high numbers, .almost 80 percent across the .state by 2012, and that was .voters opting in to get their .ballot by mail. So they were .choosing that method..And what we found with a ballot .delivery model -- some call it .vote by mail, vote at home, any .of these reforms, they need .systems to support them so that .voters know what's going on with.their mail ballot. So we .developed ballot trace. .What our goals were to enhance .customer experience, proactively.communicate to voters, create .operational efficiencies that .would save money, and then .increase the transparency and .accountability for how mail .ballots are delivered through .the post office because, if .you're an election official, you.know that, if you send out a .vote by mail ballot and you .don't have ballot trace or you .don't have a system, you dropped.it at the post office, and the .next time you see it is back at .your door again hopefully by .election day, and there's no .accountability in there in .between. So we've basically .solved that problem with ballot .trace. .So here's a slide that shows .the impact of ballot trace and .why it .has been so significant for us. .So 2008, and this was back when .we had .polling places, early voting, .and.vote by mail, and we had .permanent vote by mail at the .time. We had 1 in 5 voters who .participated in that election .had to call us to ask us a .question about vote by mail, .registration, polling places, .whatever it was..And it was a huge phone bank. .50-plus people had to answer .these calls for weeks prior to .the election. Significant .number of people had to call us,.and that is just ridiculous that.anyone would have to reach out .to their election official to .get this information. . So after that .election, we said, all right, .let's analyze all of this call .volume and data and why people .called us. Specifically, what .were they asking for? And what .rose to the top -- and it's .because we had permanent vote by.mail had increased, was most of .the calls, our top call driver .had to do with mail ballots. Am.I on the list? When are you .sending it to me? Why didn't I .get it yet? Where is it? Did .the carrier give it to somebody .else? And then all on the back .end, did you guys receive it? .Because it's election day and .I'm worried, and I want to make .sure you received it. . So we said, okay, .we're going .to develop a data system to .address this and then .proactively push out the .information..So we created ballot trace in .2009, pioneered, designed .everything in our office, .started rolling it out, and then.by 2012, 1 in 15 voters called .us instead of 1 in 5. So we saw.a tremendous improvement just in.a 2 1/2-year period from when .ballot trace was implement ed, .and we didn't have many people .on the list yet. . So then we implemented.our new law, sort of in the .middle of this slide..So 2013 is when ballot delivery .to all voters came in..We fast forward to 2016, and we .served 100,000 more people in .2016 than we did .in 2008, and our call volume was.down 95 percent..And most of it is this ballot .trace system.. Our primary, which was.June 26, we're at 1 in 60 voters.calling our office to get .information..So our call volume now is down .probably 96 percent from what it.was in 2008. This is the .operational efficiency and the .cost. We don't even have to .hire significant people to -- .significant numbers of .people to run the phone bank .anymore to .basically push the information .to voters..This is sort of how it grew. In.2014, about 15,000 users on it. .Now as of today, we have 45 .percent of our voters that are .on ballot trace. So almost half.of the voters that vote in the .city and county of Denver, we .push them a text message or an .e-mail saying your ballot has .been printed. It's gone to the .post office. Now it's out for .delivery to you. And then all .the way back then, they get a .confirmation we've accepted the .signature and it's been counted.. We also notify them if.their ballot is undeliverable. .We say, hey, it's undeliverable..Click here and update your .address or call us, and we'll .advise you where to go. We also.use it as a tool to remind them .about the election. So a week .out, we say, hey, you haven't .returned your ballot yet. Don't.mail it anymore. Make sure you .drop it off. And then we do it .five days out. And then we do .it the day before election. And.then we do it on election .morning. Anybody who has not .submitted their mail ballot, we .send them a reminder via text, .and this is all through the .system..So this is literally no work by .any .staff member, but we're doing .all of .these proactive communications. . So this is 2016, and .we're in the process of .analyzing this again. But the .power of Ballot Trace has an .administrative and operational .component for us. It has a .customer service benefit to the .voters. And we now have .determined it is a turnout .driver as well..So voters that are on Ballot .Trace have a two-point higher .rate in turnout than voters who .are not, and they have a 7 .percent higher rate of their .mail ballot being accepted. .Why? Because we notify them if .they've got a signature issue or.undeliverable or any of those .things. So voters are having a .better experience with voting .because of a system that is .customer service based but also .drives operational efficiencies..So we've made this part of our .strategic marketing. We're .constantly encouraging voters to.sign up. Even if they get that .mail ballot envelope and we've .already gotten it out to them, .we still want them to sign up so.they get the confirmation that .their ballot has been counted on.the back end..So we continually push that .message out. .This is a little hard to see, .but I just want to show one of .the dashboards in the system. .Customer service, communication .for voters, turnout driver. .Administratively, when we go .drop our ballots off at the post.office, we know .by the next morning, by .precinct, by zip code, by ballot.style across the city, the .success rate of scanning that .the post office has processed. .So we didn't know any of that .prior. It was a black hole. .Drop them off, and then we see .them later..Now we know literally down to .the precinct, probably even .routing code for .the post office, what the .deliver y success rate has been..So if we see anything that's .lower than what we expect to see.this morning, we notify the post.office. Hey, where are these .things? Why /-RPT these done .yet?.At first the post office .didn't like the system so much, .but now they do because it's .also increased confidence in .their services for voters. So .this is one of the dashboards, .and we have plenty of reports..We do GIS work to kind of .analyze delivery rates across .the city and all of that as .well. .So here's the final quote. MIT .scientist -- Charles already .gave his own quotes earlier in .the panel..So I didn't have to put him in .the slide. Did Andrew Mcafee up.here. The world is one big data.problem. I'll insert elections .in front of world. The .elections world is a big data .problem, and I think this is a .good example of how we can build.systems that .better support voters, great .operational efficiencies, and .just make the overall experience.better for all of those that .participate in this process..[APPLAUSE].>> I'm.ray Murphy, and I'm with .Pennsylvania voice..One is to let you know how we .use the data that you produce .through the state voter files .that are available to all of us..The second aspect of what I want.to deliver on today is the fact .this data allows you just as .many options as me, who are out .on the ground trying to engage .voters, this data offers you .just as many opportunities to be.target ed and specific in the .way that you communicate with .voters and hopefully save you .time and money in getting out .messages that you need to .deliver to voters. . Quickly, before I dive.into all that, I just want to .talk about Pennsylvania Voice .for a second..We are a nonpartisan statewide .coalition of organizations .that's devoted to increasing .voter participation, especially .from some of the organizations .that are the most -- or the .least likely right now to .participate. So that means we .literally coordinate the .activities of organizations in .the state that run voter .registration drives on street .corners with paper, and we are .now running as much voter .registration .in Pennsylvania as we can on .tablets, thanks to the .Department of State and the .leadership of many election .officials here in Pennsylvania..We have the most sophisticated .online voter registration system.in the country right now, which .allows third party organizations.to set up their own accounts .that allow them to collect data .about registrants but also .automatically .transmit it to the Department of.State. .In addition, our coalition .recognizes that all of the work .you do registering voters and .turning them out to the polls .and educating them about the .civic engagement process, which .we do in about 18 counties .across our state, all that work .is meaningless if at the end of .the day there are difficulties .accessing the ballot itself .and then also we recognize that,.if the way the legislative maps .in our state or any given state .are drawn don't .accurately reflect the kinds of .communities people live in, then.they're not able to elect people.that represent them. .We're part of a network that's .the state voices network. And I.name that because we have 25 .partnerships across the country,.and up here is a list of those .states for those of you who want.to check out where my .counterparts are even where you .live. Even beyond those states,.we offer .access to a sophisticate ed data.file which includes information .from data voter files which .allows folks to use data in a .really sophisticated way to .reach out to voters. .All of our network shares the .goal of fixing fair .representation ballot access, .civic participation, and we also.do work to strengthen the .organizations that serve our .communities. So one thing that .comes up a lot and I hear from .our visionary and lovely .election directors in .Pennsylvania is that a lot of .voters don't seem to know the .basics of the system, and one of.our .jobs is to try to educate them .so that burden isn't solely on .government, but that folks who .are being motivated to vote come.in with some knowledge of what .the rules are and what the .process is. . Similarly, we are .really in .the midst of a multi-year .campaign right .now to modernize Pennsylvania's .election laws. Pennsylvania is .one of only nine states in the .entire country that .doesn't offer any of the basic .voting opportunities that have .become standard in so many other.states..Not only do we not have no .excuse absentee voting, we don't.have a form of early the ./SROEing, preregistration for .youth, automatic voter .registration. All of the things.we've talked about in other .states are not available here. . Unfortunately, as we .all know in the current climate,.conversations about .modernization of our election .systems are taking a backseat .somewhat .to security, including real and .perceived threats. And so for .us, what one of our most .critical jobs in the next two .years is to work very closely .with election directors in the .Department of State to secure .all of the funding that is .needed .to build a secure and modern .voting systems..So that includes new voting .systems .but state filed voter databases .but also .things like pollbook which .improve the accuracy of .operations. So that's where we .are in our fight. .But I'm going to talk for a .second about what we do in the .context of data..So we think of data as a means .for more sophisticated .communication..It allows trusted messengers to .find .and talk to exactly the .communities with whom they have .the most currency, with whom .their message is the most .listened to. We also use data .to identify prospective .volunteers and supporters for .some of the door-to-door work we.do encouraging people to vote. .Some of the phone banking that .our partners engage in..We also use data to identify .areas of different counties .where the voter registration .need is the highest. Typically,.most civic engagement programs .do not do door to door programs .to register voters. It's .typically too inefficient. But .what they do do is find public .spaces where there is a large --.there's likely to be a large .propensity of .unregistered voters, or in the .case of a state like .Pennsylvania, which doesn't have.a fast growing population, often.the work we're doing is about .re-registering voters, .especially lower income voters .who tend to move a lot and .constantly need the registration.status to be updated in order to.be on the polling book at the .polling place closest to them. . We also know from a .lot of resource that there's a .significant number of voters who.just don't vote without a .reminder. They just don't vote .unless someone asks them to. .And our organizations that .belong to our partnership are .effective at making those kinds .of reminders. .For us, data removes some of the.guesswork. There was a time, at.least in the .grass roots organizing world, .you just .ended up going to every door or .stood on every corner or put .Flyers through someone's .mailbox. Data takes the .guessing out of the picture, so .you're able to hone in on the .people who most need to hear .from you. .There is a very specific set of .tools that we use and that we .offer to partner organizations. .So when you're dealing with .voter .registration campaigns or other .civic enphagement communities, .these standards and practices .are typically .the things on those of the .partisan side of the line as .well..They may be a little bit .different. We start with .something called the civic .engagement file. It starts with.the voter file in a given state..Then we take that file and work .with a data vendor who enhances .it and overlays .on top of that data file .consumer graphic information .that's either publicly available.or publicly available for .purchase, and it adds things to .the file that let you know a .little bit more about who the .voter is. .It can give you a more up to .date address. It can .standardize the way that phone .numbers and other pieces of data.are used. We also overlay on .top of those records census .data..You're able to identify an .individual voter that lives in a.particular census tract is .likely to have this income or .this ethnicity or likely to be a.homeowner or not. And then .last, data scientists and other .mathematicians create models .that go even further. They're .algorithms essentially that .take a whole bunch of Taye of .data factors that are able to .predict a certain inclination .that voter is likely to have. .That could include likelihood to.vote in an upcoming election. .For the partners we work in an .engagement front, when we do the.get out the vote work, we focus .on voters who have a 30 percent .to 70 percent likelihood to .vote..Typically, campaigns work on .voters with higher propensity to.work. They already know someone.is going to vote. Their job is .to push a particular candidate. .Our job is to find the folks who.are unlikely to vote or .participate in the process at .all and give them the .information they need to be able.to participate. . These models also .predict things like race and .ethnicity. We apply our own .model to the voter file to .predict what we think is dead .wood..The state and counties are bound.by very clear laws about who can.be removed from the rolls for .legal processes, and that's .important. But for our .purposes, we want to make .guesses about who's not likely a.live voter because they haven't .gotten to the point where .they've been removed from the .lists. . And models also allow .us to predict where a particular.individual might stand on any .set of issues. For our .organization, some of which are .focused on environmental rights .or .some might be focused on .criminal justice reform or .another might be focused on .raising the minimum wage or .something like that, these .models allow our partners to .find the people who are most .likely to resonate with the work.that that organization does. .Over time, that allows an .organization to build a .relationship with a voter, .and that relationship, we think,.can .ultimately be creating more .lifelong voters rather than .folks who are only .intermittently involved. .I think I covered many of these .things already, but we typically.use this data to either predict .individuals who need to be .engaged around a particular .election, individuals who live .in an area where there's a high .voter registration need or .advocacy. I go through this .level of detail, to some extent,.not just to let you know what .we're doing on the other side of.the work that we're all .collectively engaged in, which .is making sure that people are .able to participate in the .democratic process, but some of .the things that I'm saying can .serve as models for election .directors in states and others .who want to have a little bit .of help being more specific in .their outreach. .For instance, when you look, .there are a number of issues .that come up frequently. There .are polling place changes..There are new laws that come .down that needs to be .communicated to voters. For .instance, first time voter ID .laws or something like that. .There's any variety of things .that happen. There's some .obligation on the part of a .county or an election or a state.election department to .communicate to .voters, but these kinds of data .tools allow you to be more .specific in your outreach. . What specifically that.could look like, we, for .instance, actually have .contracts with data and software.vendors who literally make it .their job to provide these kinds.of tools. Those kinds of .contracts and vendors are open .to you all as well. There are .also vendors for one-time .programmatic use. You can call .a data vendor and say I .need to do a telephone town hall.tonight to everybody in X .district to explain the polling .place change in this cycle. And.you all, many of the states and .counties create the data sets we.work with. In my experience, at.least in Pennsylvania, there's .been challenges, actually .looking at that data usefully, .because there isn't necessarily .the software on hand to make it .easy to parse that. But you do .actually have much of the same .information that we then .ultimately enhance and use in .different ways. . So, again, the purpose.of this conversation is just to .remind you that there are a lot .of options for how one .can use data in a variety of .ways that really address .day-to-day concerns. . I think, with that, if.you have questions or want to .talk about what options might be.available to you .in terms of this target ed data,.I'll be happy to talk with you.. >> We'll have .questions and answers a little .bit later. Next, Laura.. >> Okay..Do I have to click anything, or .does it just come up?.>> They'll put it up.. >> There it is. .My name is Laura Herzog, and I'm.from Hendricks County, Indiana. .Our county is one county west of.Marion County, which is where .Indianapolis is..We have approximately 111,000 ./REPLG /* registered voters now..We are the second fastest .growing county in Indiana right .now..So it's critical for us to keep .on top of gathering data to keep.things as efficient as possible .for our voters..So I feel amongst friends I want.to share this with you. .This is my first time in .Philadelphia. So I had two .goals..One is to get a real Philly .cheese -- whatever. Steak and .cheese..And the other was to see the .Liberty Bell..So I thought when vice chair .Christy .McCormick earlier talked about .election administrators having a.touch of OCD, .that's true because I did make .it to the Liberty Bell, and it .still has a big crack in it. .And I thought, haven't they all .heard of flex seal? Anyway, I'm.enjoying my stay here. .I'm here to specifically talk .about electronic pollbooks..I do want to let you know that.our county went from precinct .based voting to a full vote .center transition .just this last primary, which .was May 8th..Before, we had 104 .precinct-based polling sites. .We went down to 26 vote centers,.reduced our poll worker force .from 520 .down to, I want to say, a little.over 100..So it's hard to talk about .electronic pollbooks without .talking about vote centers..They go a little bit hand in .hand..Peanut butter and jelly, .biscuits and gravy. You're .going to go crazy now. Sonny .and Cher. .In Indiana, as of April 6, there.are 60 of 92 counties that are .using electronic pollbooks..35 of those are vote center .counties like mine..25 are non-vote center counties..Like I said, our first election .using .this was just last May. .Currently, we have four e-poll .book vendors that are certified .in the state of Indiana..If I could talk a little bit .about that..Indiana is very aggressive, .thankfully, on their e-poll book.certification..Each vendor and county must go .through a really thorough .acceptance training program. .The counties and vendors must .comply .with before using in an election.certain .specifications are met to ensure.correct functionality. We are .also fortunate enough to have an.organization called VSTOP, which.is a voting system technical .oversight program..They are out of Ball State .University. .A quick background for us..In Indiana we do have a photo ID.law..So on our electronic pollbook, .which .are iPads, the voter must show .their photo ID. Typically, it's.an Indiana issued driver's .license, which makes the .check-in process extremely quick.and efficient..Bar code is on the back of the .license, the iPad camera will .access the bar code, and an .electronic keypad comes up for .the voter. They sign it..They're issued a valid ticket, .which .then goes to the voting machine..It's an extremely quick check-in.process, where before with the .bulky pollbooks it would take a .long time for .the workers to find the right .letter of the alphabet.. Then we have the .father that signed on the son's .line. So all those efficiencies.have improved..We can monitor in realtime on .election day lines..I'll have a few screenshots for .you, .functionality of the electronic .pollbooks. Do they have full .connectivity? Are their .printers synced? Are they at .the proper charge level? .Location activity is extremely .important and helpful to us. We.can communicate directly with .the poll workers through the .electronic pollbooks. Certain .things like, hey, if you have .a little down time, don't forget.to sign all of these forms that .need signed, that sort of thing..And, of course, we can monitor .turnout in realtime. .This is a really cool slide. .This is our dashboard. You can .see on the left those are all .little subtitles that you can .open up and expand. This slide .in particular kind of gives you .a really nice view of what's .going on..It shows you obviously that your.polling places are 100 percent. .You can see how many have voted .absentee, how many have checked .in, what your percentage is..Ignored 70 critical iPads .because this was taken just last.week. . Okay..And this slide, this will give .you individual vote center .activity..It will show the number of .e-pollbooks that are at each .location. The number of people .that have voted..You can see here at the.Amo community center, we had 195.people check in and vote..Below that at the aVon United .Methodist church, we had 1 /* .2,161 voters check in. That .data is going to be imperative .for us..Just so you know, the Amo .Community .Center only has three voting .machines, where the Methodist .Church has 15. So this is going.to help us figure out .what locations we need to beef .up poll workers and machines or .maybe in some cases combine .smaller ones together.. This is a good slide. .This will show you turnout. Our.polls are open 6:00 a.m. to 6:00.p.m..and I believe this slide is a .repeat of the others, so I .apologize. It's still pretty .cool to look at..And these are -- this is a .report .that's hard to see, but it's .produced by the electronic .pollbook..You can generate oodles and .dozens of reports that are very .beneficial. . And then the fabulous .gals in .my office, Tammy and Jen, they .turn this information into pivot.tables..This is, again, more data that .we find very useful..We also can tell are the voters .voting in their own .neighborhood?.Or are they voting on their .commute to and from work or .taking their son or daughter to .soccer practice? It's all very .interesting to look at..An unexpected benefit from .transitioning into poll books, .this isn't my storage closet. I.borrowed this off the Internet. .This looks like what our storage.closet would look like before, .just paper and paper in bulk..Before, we would have to with .the poll book scan each voter's .signature. Now with the .electronic poll book, it's done .with the push of a button. We .went from this to that. That's .our entire primary right there..Provisional ballots, very .unexpected..Average for this past election, .125 to 150. We had one. One .provisional ballot? Why? .Because no one was in the wrong .place. . Okay. I want to tell .you quickly about our poll .worker that you all know her. .You all have one..She has seen all of these .changes in her life..She wasn't ready for this new .contraption called electronic .poll book. It's for young .people. It will never work. .Through some good training and .talking, Dimple tells us now .it's the best thing since sliced.bread. You all know her, don't .you?.And then Dimple got her friends .to come join us. And whoops. .So I just want to say that it's .efficient. The budgetary .savings to the county .just in printing costs alone is .substantial. .My time is about up. I just .want to say that it's very .efficient. There's lots of data.there. And thank you all for .having me.. >> Last but not least,.Ms. White..>> Thank you, Tom..Today I'm going to present a .lesson on how to lessen the line.on election day. This was .experienced from the .presidential 2016 election..So this is a shot from .presidential 2012 in one our .precincts. Unfortunately, in .Prince William county, the 2012 .election didn't go well. There .were lines three to four hours .reported in many places. But I .use this photo kind of as proper.motivation, if you will, to .prepare for 2016. .So next there is a congressional.hearing held afterwards to .determine what went wrong in the.2012 election..As you can see, the last ballot .was recorded at almost 11:00, .which was four hours after polls.closed. Polls close at 7:00 in .Virginia. So next we took some .time to figure out what data we .should be looking at..We look at the demographics of .Prince William County. We study.the number of voters who utilize.absentee voting and which .precincts were the largest. . So we were on a .mission to fix this problem. So.we looked at some other things..Statewide,.forgive me..2012, we had 255,000 voters. We.went up to 274,000 in four .years. At that time in 2016 the.state had deployed an online .voter registration system. So .we were seeing more voters .register. So we were getting .ready to see a huge uptick in .September and October of 2016..Now, as of July, there are .279,000 voters in Prince William.County, which means we have to .rinse and repeat, which means in.the election result we have to .reproduce the result we had in .2016 for 2020, but with more .voters in the picture. .So looking back at the history .of absentee voting in Prince .William, we were trying to .figure out why voters were not .taking advantage of absentee .voting..As you can see, it dropped from ./* in 2012 from 2008 to only 10 .percent to 12 percent of voters..So the solution became to inform.voters on how to lessen the .election day line by voting .absentee. Thus we began a .multimessage inform voter .marketing plan. We were all .over the place and very scrappy..So the goal became 68,000 .voters absentee in Prince .William, which was to make 25 .percent out of the lines on .election day. This was a lofty .goal, but we had to accomplish .it for a better overall .experience for voters. . Next we looked at .commuters, .and we discovered that almost .150,000 folks commute out of .Prince William .County, and commuting is a -- or.having business outside of .Prince William County is a .qualifying reason for voters to .vote absentee. Let me back up .and explain for a minute..In Virginia, you must have a .reason to vote absentee, meaning.you have to be eligible by .certain reasons in order to vote.absentee or early in Virginia.. So 68,000 voting .absentee was about half of the .commuting population in Prince .William County. So, again, we .wanted to shift that to .absentee. So we set our goal. .Then we realized we had to kind .of go into myth busting mode. .We realized that voters think .they have to qualify or fit a .certain criteria to vote .absentee, which is a true .statement. However, we got the .sense that voters didn't know .the number of qualifying reasons.that would entitle them to vote .absentee. So we mailed a voter .guide to every .household in Prince William .County using .every door direct mailing with .USPS. This is the same method .in which you get your Chinese .takeout menus. But we got one .to every single household, and .within that, we included a .sample ballot and six things .that all voters need to take .care of before election day. .Another myth we had to bust was .that absentee votes are not .counted. We made it clear in .every election piece we put out .there that by law all absentee .votes must be counted. So we .had that in every place we could.tuck it. .Here's how we focused on getting.the message to commuters. We .set up electronic boards in all .the commuter lots. I think .Prince William has about 15 very.large commuter lots on 95 and .66, heading into D.C..We placed ads in the commuter .train magazine..We placed ads in HOA magazines .where folks were centered and .where they were living..And the Virginia department of .Elections also helped us out by .placing signage on buses and .additional signage in the .commuter lots. . So the photo is not so.hot, but this is what the .message board looked like..We also set up a very .inexpensive texting program..It said text PWCvotes to 94253 .to get voting updates and .reminders. Wherever we could, .we had these places at .intersections and public places .where we were allowed to do so. .So we really went all out to get.the message across. .So back to the congressional .hearing, some who testified .Monday said pregnant women were .forced to stand for hours. .Again, this is an indicator that.education and information about .absentee voting was not getting .out there. So then we went on a.mission to create some trading .cards. Here you go. .So we had some fun with it..My team is scrappy, and we have .a .sense of humor as all election .officials must have..So we said did you know by law .all absentee ballots must be .counted. . This is what we did. .Too old..Some folks took issue with this,.actually that were younger than .the set .we were trying to get the .message to..These cards got returned..The older folks who got it said .they also really liked it. The .older folks we were trying to .message say they loved the card..But folks in that next tier of .age were a little offended..Here's the various ways to .commute in prince William .county. First responders can .vote absentee. And we made sure.that first responders knew they .did not have to be on the .schedule for election day as .long as they are first .responder, they can vote .absentee..Active duty military, we also .went after..I'm sure you all know the show .Quantico. That is in prince .William county. . Here's the back of .every trading card. So we made .it clear, again, what the .deadlines were..Last day to vote absentee, how .to get a hold of us. So we .crammed as much information as .we could there. .So next I have -- we did some .scrappy little homemade with .found items videos, if you will..[Dog barking ] .[Sirens ] . >> So we put those all.over Facebook, and it was an .inexpensive and cheap way for us.to get the message out. We .relied on other voters to get .the message out to their buddies.and friends. .The next slide, data does not .lie, and getting ready for this .present /AEUG, I did not realize.that VPAP had done a lot of my .work for me..So Virginia public Access .Project is a nonpartisan group .out of Virginia, and they took .data across the state and did .some visualizations for me, .which I will show you here. . So it's hard to see. .I believe in the marriage .between data analysts and .graphic designers, which you've .seen plenty of examples today. .Here you can see where Prince .William County led the state in .early voting or absentee voting .for 2016. So we're that very .first blue bar right there. .Very proud of that. Hope we can.continue..Next slide, so here's what .absentee voting looked like in .2016. If you go back to 2012, .the next slide, you see where .the dot moves. Okay..So I like to toggle back and .forth, forward, back, see how .far we went? Oh, that's 2004. .That's good..Thank you for helping me. . So all that being .said, I really love playing with.this data for Prince William .County because you can .really see the shift and what we.did to make election day work .and get folks to vote absentee. . Next slide would be .this one. Reasons for early .voting in Virginia. Like I .said, we studied, and we knew .the demographic we needed to .target commuters. And .apparently, we were successful .in doing that because the .biggest block .that vote absentee were work .conflict .voters, and we had 16,000 .ballots cast based on a work .conflict..So I also found this data very .interesting. .So we know our messaging worked,.and now we know it's .quantifiable, which I like..Again, back in August of 2016, .the .state launched its online secure.portal, which meant that we were.taking online voter .registrations and folks were .able to vote -- /TOER give me. .Register to vote through a .transaction at the DMV. So we .noticed that more and more .people were using the portal .than to register to vote through.the DMV. So we're paying .attention to that. Everything .that we put out there had the .citizen portal information on .it. So citizens were able to .check their voter record, update.their record, confirm their .polling place, also apply .for an absentee ballot, and see .their sample ballot. So we made.sure this was published .everywhere possible. .With our voter information or .education, we wanted to make it .clear that voters have some .responsibilities to address .before election day. So to know.that their voter .registration is up, to bring .photo ID, what will be on the .ballot, what your polling place .is. Last two that you'll be .marking your ballot -- your vote.on a paper ballot and then .putting it into new scanning .equipment. So optical scan was .new for Prince .William, and we knew we had a .lot of cicada voters..Cicada voters are only those .that vote in presidential, every.four years. So we had a sense a.lot of them didn't know they'd .be using new voting equipment .and marking information on a .ballot. . Finally, we wrote .again, election day..Avoid the long line by having a .ballot mailed to you. .So on election day, shaving .seconds. My approach was .similar to Neal's from Orange .County. I knew that there were .things we could tweak to make .things better..I also paid very much attention .to the elbow of death provided .by the BPC and MIT. They were .partners in helping me get .through presidential 2016. I .listened to everything they gave.me, so thank you for that. We .looked at our line data .collection, and we prepared our .precincts to be ready for a line.at 6:00 a.m..We provided those folks with .line chaser tablets. So .basically, a line chaser tablet .took our voter registration .list, put it on a tablet that .was easy to view. It was not a .check-in tablet. And we asked .the high school pages to make .sure the voter ID was out before.getting to the check-in table. .So those were shaving seconds. .We provided more officers and .laptops in larger precincts. .And instead of using a voter .permit, we gave the ballot .directly to the voter..So we got rid of the second line.there. .Next slide, we provided a .majority of stand up marking .booths. This allows the voter .to move more quickly instead of .having to sit down and mark the .ballot. I am considering .switching to an official felt .tip marking pen instead of a .standard ball point..I think that will also shave .some seconds off. .In larger precincts, we put in .more equipment, very simple..And we also implemented a chief .headquarters connected to our .emergency operations center, and.we used radio communications in .every single precinct along with.my office. So we did it..No voters in line at 7:00 p.m. .in November 2016, and all 91 .polling locations closed on time.in Prince William County. .This is my crew. I'm very proud.of my staff. We are a team. .We're scrappy. We get along .well..We're just a slightly crazy .bunch but in a good way. .And finally, that concludes a .lesson on how to lessen the .line. Thank you. .[APPLAUSE] . >> I want to thank the.panelists. I think this was an .excellent panel. I learned a .lot myself today. How much time.do we have for questions? Half .an hour..So I'm going to take up half an .hour of time asking questions..Kidding. Jokes for those folks .at home. I just want to make .sure people are still awake .after /HRUFRPLG..I know that I see some heads.lunch. I know that I see some .heads nodding. .Reducing wait times at the polls.have been a big focus for the .election community since 2012. .We heard a lot from Michelle .about data collection in .Virginia and elsewhere to try to.tackle this issue. How are .these effects playing out in .your counties and states?. >> Well, for Colorado .-- and I'm going to make a .request for the audience..Just especially based on the .wonderful presentation from .Prince William. But Colorado, .we mail out a ballot to every .voter proactively before every .single election. So there's no .question..That's their noticity /*. They .get their thing. They don't .have to give us an excuse. And .I was watching the presentation .and I'm thinking, who doesn't .qualify for one of the excuses?.I'm starting to think, well, .clearly, that's one of those .points that just confuses .people. .And then just the term absentee..I don't know what others think .about this, but most don't know .what that means to say absentee .ballot..I would love to eliminate .absentee from terminology all .together with this because I .think voters don't necessarily .understand it..I think that would be a great .thing long term is to strike .absentee out of the dictionary. . But for us, because we.do mail out ballots, and we also.have same day registration and .vote centers. We do have in .person voting options preserved .for a two-week period, and we .build up to election day and .have a certain number of vote .centers across the city on .election day, and you can .register and vote right on the .spot, and .there's no deadline, if you .will, to register to vote. What.that means is our vote centers, .especially on election day, and .we've collected this data for a .long period of time, we get hit .very hard with election day in .Denver with sort of the lovely .people that were too busy for .three .weeks to come in, and they are .there on election day, which is .totally fine. We just now have .the data to expect that. .So we're building up more .systems..We've adjusted our line .management strategy..For this upcoming election, .we'll have wait times and all of.that available online, if there .are any. But we're basically .95-plus percent of .our voters use the mail ballot .we mail to them..And they -- you know, the .feedback we get by getting your .ballot at home is I had time to .research and vote it at home. I.spent a few days talking to .friends about what they were .going to do. I've heard people .say they had wine and cheese .parties and talked about various.issues, and then they'd all go .off and vote their mail ballot .at home. .But the interesting thing is -- .and this is why I call it ballot.delivery and sort of voting at .home because.it isn't really vote by mail. .And the real reason is we have .85 percent of our voters that .use their mail ballot drop it .off in person to one of our .Dropboxes, drive up dropoff, .vote centers, or any of those .locations. They're not actually.mailing it back. We're seeing .less and less people mailing it .back. So vote by mail is also, .for me at least, an outdated .term. Really we're just .delivering the ballot out. The .mechanism is the post office. .And they're still experiencing .this in person voting option by .dropping their ballot off..So we've sort of, I think, .managed to preserve a really .nice experience for the voters .that still want that election .day experience, but they're .maybe using .that ballot they gave to us.. >> Sure..Lines at electronic poll books, .I .personally like to see a little .bit of a line because it's .showing that people are there. .Four hours, no. Anyway, it .shows that people are getting .out there and voting, and I .don't want them to get too .spoiled. But with the .electronic poll book, the .check-in time is so .significantly reduced. It's so .fast that, if we do get any .kind of a clog, it's going to be.between .the clerks table, where they .sign in, and the actual voting .machine, which we have practices.in place to not let that happen..But also on the dashboard, that .we can see realtime on election .day, we can tell what places are.getting hit. So we know what do.we need to do? Do we need to .send backup poll workers?.Do we need to send extra .workstations, more voting .machines? That's how we handle .our lines with the electronic .poll books.. >> Ray, thank you for .your presentation. I think .there's a lot of election .officials here today who have .misconceptions about .organizations like .yours, gathering data just to .sue them basically..But I think that, in listening .to your presentation, I thought .it was fantastic, you were .saying how much you work with .Pennsylvania and so forth. . Are there other .misconceptions out there that .you would like to .disspell from election -- you .have a roomful of election .officials, or partially..The things that you would like .to disspell for them..>> That's an open-ended .question..>> Or commonly misinterpreted .things. .>> I can't speak for every other.state, but I will say the sector.that I work in is probably as .complicated and full of as .different kinds of personalities.as any of your sectors. I think.the most common thing that comes.up for us -- and I think this is.applicable to many states like .Colorado and Virginia that are .considered more battle ground .states, we tend to have a .constant influx ofdifferent .kinds of personalities as any of.your sectors. I think the most .common thing that comes up for .us -- and I think this is .applicable to many states like .Colorado and Virginia that are .considered more battle ground .states, we tend to have a .constant influx of outside .influencers who come into the .state for a short period of time.to run an election, do a voter .registration drive, do .door-to-door campaigning. They .do not have the reasons or the .incentives that those of us who .are base /TPH-D this state have .to maintain relationships, have .a long view. The concept of .we're thinking about chess and .some people are thinking about .checkers. They just want to get.in and make that quick move. . So I spend half of my .time in an election year doing .traffic control with those .entities and saying things like,.it's not a good idea for you to .do the program you want to do in.the county you want to do it. .And your attitude toward people .who run elections is really not .matching the experience we have .on the ground..Like much of life's .relationships are important. .And those who work in a .particular place dig in over the.long run, and you get to know .people, and you can understand .things are the way they are. . I think it's not .necessarily a misconception, but.I don't know, speaking to the .election directors, I saw some .heads nodding, and I'm hearing .from some of these folks, there .are a lot of things we would .like to see and do in our state .that we are currently .not able to do because of our .legislature. It is not because .of a lack of will or lack of .protest from people on the .ground, and in order to change .that, we have to work together. . I think it's a .misconception I would say for .people outside of Pennsylvania, .explore who your advocacy .community is and see if there .are more opportunities for .collaboration than you might .know about. .>> Michelle, I think you did a .great presentation. One of the .things that we've done at .the EAC each year is we've had.with our executive director .Cleary Awards. Those awards are.given in various categories on .things that are innovative and .moving forward towards making .participation better and making .things easier for voters and .election officials and giving .more information out there .overall..I would hope that you submit .some of these things for those .Cleary awards when they pop up .coming forward. .And one of the things I wanted .to ask you is how can other .counties think of ways -- .because I'm a Virginia .voter, and I'm not too old for .part of .that, but I have to give an .excuse every election cycle..It's basically, you know, the .excuse .absentee balance the 0 /* -- .balloting. So are there other .things you're thinking about for.2018 and 2020 that folks can use.to make sure they're able to .cast their ballot and so forth?. >> Well, I'd .definitely like to continue .voter education although .I've decided to rename that .voter information..loting. So are there other .things you're thinking about for.2018 and 2020 that folks can use.to make sure they're able to .cast their ballot and so forth?. >> Well, I'd .definitely like to continue .voter education although I've .decided to rename that voter .information. Because there is .an implication with voter .education that you're doing .something else that folks who .are funding that may or may not .like. . That being said, I'm .going to try to put more boots .on the ground. One program I .thought of is the absentee .advocate program, where I would .take officers of election or .folks in .the community, like Erik's .program, go .out and let them know why to .vote absentee. There's like 18 .reasons that make folks .eligible, but not everyone knows.what those are. So I want to .continue on the voter .information mission. I hope .that answers your question..>> No, it does. Laura, I have .one more question for you in .terms of the e-poll books. I .think those are fantastic in .terms of shutting down long .lines and so forth because in .2012 that was one of the major .issues that we heard about.. In going towards vote .centers, and the same with .Colorado, have you notice Thad .they're -- if you don't have .this, fine..But have you noticed there's .been an .uptick in participation.through data or remains the same.or so forth? .>> Well, this was our first .election using electronic poll .books. So I don't have anything.to compare it .to really until we move further .forward. However, with that .said, the feedback .from the poll workers and the .voters and .the people involved has all been.nothing .but supportive and positive that.it can .only help improve the lines and .the efficiencies..So I think I need a couple more .under my belt maybe..Maybe I'll come back in a couple.years, let you know..>> As Amber was saying, she .participated in the first data .summit we .had, and commemorate .commissioner McCormick because .this was basically her idea to .have this happen, and we've .continued on. I'm hoping that .2019 we'll have another one and .continue on with that. Sorry I .interrupted you..>> No, you didn't. I'm done.. >> See, we have such a.polite panel..With that, I've only taken up .ten minutes of question and .answer period. So if you have .questions, we have two .folks who have microphones from .Pennsylvania who I would say, if.you can make it brief so we can .get as many questions in as .possible, that would be great. .We can start right here.. >> Gary from Dauphin .County, Pennsylvania..Like Ray was alluding to, we're .kind of beholden to what the .legislature says thou shalt not .do. There's a lot of things .we'd like to consider to .streamline elections with lines .and stuff like that. How do you.transition from precinct based .to vote centers? Do vote .centers still provide you the .ability to hold precinct level .elections or by ward elections?. So like if I'm pulling.in a couple of municipalities .into a high school that are all .part of let's just say the .Middletown area high school and .you've got three municipalities,.can I .still in that vote center can my.royal ton first, ward second, .Middletown, whatever, as a .polling place. Are you able to .do those things within the vote .center?.>> I got this one..Yes. The beauty of the vote .center is the voter can never be.in the wrong place. I for the .first time in 24 years actually .got a vote on election day. I .always have to vote early .because I'm always working .election day, and I can't drive .back home..So this past primary because it .was a primary and we had .delegates and precinct .committee, and it was a large .ballot, we had several hundred .ballot styles that were loaded .on each machine .that was deployed.out to each of the vote centers..So whether you lived in north .Salem or the town of plain field.out in the .country, your ballot was going .to be, no matter where you .stopped, your ballot was going .to be there. .And the beautiful thing that's .up and coming for us in the fall.is our .county, like I mentioned, we're .the second fastest growing in .the state right now..We have annexations happening .constantly, where you have this .nice little country county out .here, and then the town comes in.and wants part of it but not all.of it. So now we have a split .precinct. Some voters get the .town ballot and others don't. .It's so hard to manage. With .the vote center environment, the.ballot ticket will dictate which.ballot style they get..And coming up this fall, we are .getting encoders on the .electronic poll .books that will automatically .load the information on the card.that will go straight to the .voting machine so it will .virtually -- I hate to use that .word..It will remove poll worker error.in setting the wrong ballot. . Also in Indiana, I .wanted to .mention our early voting is no .excuse..We do have by mail, we have .certain .criteria.. So we have very successful .early voting. I hope that .answered your question. . >> I was going to add .to her answer, if I may.. >> Please..>> Thank you.. >> Such a polite .panel.. >> We're doing the .same in Prince William County .with our absentee vote centers. .We have three major absentee .vote centers. They've become .the same thing, and I wish we .could implement that on election.day. We have 91 polling places,.and I'd rather manage less .polling places on election day. .But she's exactly right. All .the ballot styles are available .wherever the voter decides to .show up. . One thing we're doing .in Prince William is we're using.two sets .of EPBs so when the voter first .shows up -- and this is just .absentee vote centers, not .election day. Folks are checked.in on one set of EPBs, and .they're given the application to.sign and indicate what reason .they need to vote absentee. .Then they take that to a second .set of .EPBs, which has a ballot program.into it based on their split. .So they have split precincts as .well..Some elections we've had easily .43 ballot styles to keep up .with..So ballot on demand tied to our .EPBs ensures that the right .ballot is printed right there on.the spot, which also .allows us to save.money on preprinting ballots. .So it's very possible to do what.you're trying to do.. >> And I was just .going to add, because we've done.vote centers in Colorado for a .long period of time..In fact, they started there in .Larrimore county back in 2004 or.2005. So we've done -- and then.in Denver we've done vote .centers for early voting prior .to us having this new ballot .delivery system. So it ./STKWRUFT kept transferring .everything basically to vote .centers. And vote centers does .have an impact on turnout. .You're not left out or voting on.provisional because you show up .at the wrong place. It isn't .just the vote center for that .turnout increase. For us, same .day registration was a big .driver of that. If you just .look at the top states for .turnout, all six of them have .same day registration.. So there's definitely .a correlation between making .registration easier overall .combined with vote centers, .whether it's ballot delivery or .other policies that help with .that.. >> Sheila Lany. I .live in Montgomery County, .Pennsylvania. Two things. One .thing about the long lines..It really depends on if you're .trying to track the electorate .or trying to expand the .electorate..I mean, I know there are .processes, but sometimes it has .to do with intent. . For Erin, I think it .is, my question for you, I know .we are all here talking about .data and being data driven, but .as a community person, it is .what's the face to face .interaction that is most .important about driving people .to the polls. Could you talk a .little bit about that.. >> You mean Amber?. >> I'm sorry. The .gentleman next to you.. >> Ray..>> I'm sorry. I got your name .wrong..>> Ray Murphy.. >> I'm sorry. Thank .you.. >> That's okay..Yeah, I mean, there's .substantial .research at this point on what .are the most effective things to.increase voter turnout? There's.kind of three ways that I think .about it. There's a significant.aspect of voter turnout that can.be increased by efficiencies in .the way that elections .themselves are administered. .Most of those have to do frankly.with the law, the way that the .election process is structured. .The more that you offer .convenient and accessible .options to voters, you do see .some increase in turnout..It doesn't in any way, though, .make up for the huge turnout .gaps that we see across the .country, and those get into sort.of the other aspect, which is .that face-to-face interactions .between potential voters and .trusted messengers .are the absolute most effective .way to increase turnout. .They're also the most costly and.difficult to do because a .face-to-face interaction is .fundamentally more complicated .than a phone call or a mail .piece or a text. . I think there's a .third .component, which is, you know, a.bigger philosophical question .for all of us, which is what .would it take for every person .in our democracy to feel like .participating in an election is .something that they should spend.time doing? I don't want to get.into that philosophical .conversation right now, .but I think this notion of .shaving seconds on election day .is a concept that could be .applied to the whole enchilada. .There are a lot of ways that .elections .are administered that create .unnecessary barriers to .participation..Again, though, my experience is .there has not been enough .political will at a big picture .level to make sure those changes.can be implemented fairly by the.actual election administrators .who are on the job. . And often what you see.is sort of -- and I think you .see that on this panel. There's.some people who are really going.above and beyond to be .excellent, and that's great, but.everyone should have the .resources in administering .elections to do what they can to.shave seconds in the bigger .picture.. >> I guess on this .side. One question per -- I'm .teasing.. >> This question is .for Mr. Murphy. I'm /TPHR Bucks.County. From the day your .organization decided it would .implement the third party API .with the Pennsylvania Department.of State, how long did that .process take for development and.implementation? And what were .some of the lessons learned as .the application was rolled out .with actual voters at the grass .roots level?. >> This is maybe a .process question. I know Mike .Moser is talking about the .Pennsylvania web API. Did you .already talk about it, Mike? So.Mike is really a resource as .well and the team at the .Department of State. But from .my memory, it was a relatively .quick development process. The .Department of State rolled out .online voter registration in .August of 2015, and development .had kind of maybe begun in that .spring. It was the advocacy .community that initially said, .if you're looking for a cost .savings from online voter .registration and you want the .maximum .number of participants, most .third party voter registration .efforts will not use an online .portal because it is considered .a best practice in third party .voter registration to retain the.information of the voter to then.remind them to vote on election .day..That was sort of laid out to the.Department of State, and I think.their leadership was incredibly .visionary in understanding that .that was a true statement and .the goal was, in fact, to go as .paperless as possible. . So creating an online .web portal where third parties .could push some of that data in .directly to the Department of .State would really save time. I.would defender to DoS, but I .would say in the grand scheme of.the DoS budget, it was not a .significant amount of dollars .that was spent on the coding of .that system. Some of that has .to do with the way that the .Pennsylvania shore system is .established..So I wouldn't presume to speak .about whether other states could.as easily modify their online .database to accept those data .sets. . What has been .exciting, as time has gone on, .is the concept of connecting a .third party database to the .voter registration system is not.limited to third parties. So .there are many state agencies, .particularly where it really .makes sense to connect these .things because you can drive a .large volume of voter .registration applications for .new voters .or change through a completely .electronic process. From a .county perspective, what that .means is there's no data entry. .It's more or less clicking an .approval or a disapproval for an.application for all the typical .reasons, and it just saves a lot.of time, money, and effort on a .lot of people's parts. .I hope that more states will .replicate what the Pennsylvania .Department of State has done .because I think it's also worth .pointing out, with no offense to.anyone involved, but the .technology that was implemented .is .actually quite standard in most .of the rest of the commercial .world, and it's been in place .for so or 20 years. We're .always running a little behind .in government in terms of .catching up with tech trends, .but the idea of sharing this .kind of data is quite .simple to do and has so many .benefits..>> I'm getting the wrap-up .symbol even though my time says .20 to..So question on this side? No? .We'll go back to right here..>> On the idea of early vote .centers and things like that, in.Pennsylvania, if you vote by .absentee, you can then change .your .mind, go to the polls on .election day. They will .physically mark void on that .ballot. It's not open. It's .not counted. And you get to .vote your new choice on the .polls. In your early vote .centers, you're talking about .voting on machines and stuff .like that..How do you unring that bell if a.voter changes their mind before .election day?. >> It's my .understanding that's a fairly .unique process that you have .going on in Pennsylvania..You can't pull your ballot back .once you've submitted. We start.processing three weeks out. .Even when we had early voting, .early voting is on equipment. .So once you're cast, you're .done. And the only way you .would then be .able to go back in is if you .voted on a provisional, which .then wouldn't be counted. So .there's no rolling back once you.submit? . >> Microphone .microphone..>> Yeah, and /* Colorado is a .swing state. Yeah, you've got .to know. I think in most .states, bringing it back is not .something that occurs. So yeah..You've got to pick.. >> We had a lot of .them change their mind in '16.. >> I think we have .time for one last question. If .no one's going to ask. Come on..This is your opportunity to get .a question in..If not, I will finish with my .question. People are walking .out too. . We talked about .electronic tracking, electronic .poll books, enhanced voter .files. None of these tools .existed, at least in its current.form, when the EAC was formed 16.years ago. What can we at the .EAC do to keep encouraging .innovation in this space? And .what types of data driven .practices and tools do you think.we'll .be talking about in in the near .future?.If each of you want to answer .within a minute or so..>> I can..Yeah, I'll get it done..So when I talked about the .encoders, I know they've been .out there a while. I think .that's something we're all going.to see..And then one misconception.that a lot of people, mostly .candidates, have about .electronic poll books has .nothing to do with the question .you asked, is that they're going.to get results in faster. They .just think, oh, we're going to .get numbers quicker. No. So .just throwing that out there. . >> Anyone else?.>> Yeah, from my perspective, .I mean, our entire approach in .Denver .has been analyzing the data we .have, but also continuous .improvement. So elections are .very much people, process, and .then sort of the concept of .continuous improvement. And you.can only do continuous .improvement. You can only .continue to get better if you're.curious about what is happening .currently in your jurisdiction. .How to analyze that data and .then build systems that better .support the voters. .For me, I think doing what's .right by way of the voters as .sort of .the first premise and step is .the right approach in elections,.regardless of party affiliation,.regardless of where they sit. .Everyone deserves this fair, .equitable process across the .board that's consistent. So any.of these like inconsistencies .that exist -- you know, I mean, .the sort of absentee with .excuses and all of these sorts .of things. Like the confusion .that that creates .for people that do this once a .year, if that, maybe once every .four years, is sort of, I guess .in my mind, and I would say .disrespectful frankly to the .voting franchise. This is our .world, but it's certainly not .something that they do. .So I just -- I'm a big .advocate of being curious and .creative .about solving the greatest .challenges we have in elections..That's why everyone up here has .provided different examples of .improvements..So to me, I think the election .assistance Commission continuing.to highlight the best practices .and the good things and having .events like this to facilitate .the conversation is really how .we all as a collective get .better.. >> Thank you..So with that, I want to thank .the panelists. I think this was.an excellent, excellent panel..We'll now break for about 15 .minutes..Just come back at about 3:15 for.our .last., last panel of the day, which .will focus on the post-election .period. I want to thank my .panelists again today. .[APPLAUSE]. [BREAK] . >> Can everybody .please take a seat. Bring the .chocolate with you.. Hi, everybody. Our .panel decided you guys have had .enough data for the day. So .we're going to do a bunch of .show tunes. Is that okay? . Hi. I'm Kathy .Boockvar..I'm senior adviser to Governor .Wolf .here in Pennsylvania on .election..I cover elections with the .Department of State and with .county election officials across.the state as well. I want to .thank again the organizers of .the event. You did a great job .over not only the substance of .the panels, but one of the .things I really loved was the .diversity .of the people who were .presenting because it's really .-- the data is fantastic. But .if you can't translate the data .to the poll workers and the .people on the ground and the .voters, it's not going to go .anywhere. .So I really want -- I just want .to say thank you and have .everybody .give a round of applause to .everyone who presented today. .[APPLAUSE]. So this panel is about.the end of the day, the end of .the election, post-election. .Most voters typically care about.the results of the election, but.election officials and other .data experts know that we need .to use tools that we've gained .from throughout the day and .afterwards to make each election.better..And so this panel is made up of .diversity of backgrounds who are.going to tell you what they've .done in their states and what .they've done in their .professions..So I'll start with .introductions. .So we have Rey Valenzuela, the .director of elections in .Maricopa County, Arizona..And Rey has been with the .Maricopa County elections .department for 28 years. So you.probably know a thing or two..And has been the election .director since April 2017..And Rey is a graduate of the .election centers from Auburn .University and also Maricopa .County management institute .graduate, and is also a .Secretary of State certified .election officer since 1996. .Rey has served for the past 12 .years as one of two Arizona .representatives on the EAC .standards board and in 2018 was .elected to serve on the .executive board and currently .served as secretary. .Next, we have Jennifer Morrell .from the election Validation .Project the democracy fund. .Jennifer is an election .professional, and she's .partnering as a consultant with .the democracy fund to lead this .project aimed at increasing .trust in elections through .rigorous audits, standards, and .testing..Previously deputy director of .election ins Arapaho County, .Colorado, Jennifer was .instrumental in Colorado's .successful implementation of the.first statewide risk limiting .audit, and she's going to tell .us all about that today..Jennifer is also an election .center graduate and holds a .Master of Arts in management and.leadership and is also on .the advisory committee for the .center for civic design. .Jennifer is also a veteran of .the air force. Thank you. .Next to me on my left is Kara .Rahn, who's the director of the .department of voter services in .Chester County, Pennsylvania..Kara has 19 years of .organizational communication, .operational, and management .experience, and has enjoyed .success in government, .corporate, and not for profit .arenas. .Kara prior to this was at .Axelon's nuclear generating .station. I'm interested to here.how that transfers into .elections.. >> More than you .think.. >> She served as the .deputy director in Delaware .County transportation management.association .and held several legislative .roles for PICO. Kara earned her.Bachelors in communications and .public relations and .earned an MBA from Eastern .University. .And last but not least, I have .Bridgett King, who Bridgett is .an assistant professor in the .department of .political science and master of .public administration program at.Auburn University..Bridgett teaches graduate and .undergraduate courses in state .and local government, American .government, the presidency, mitt.Cal participation, and public .policy..She's also an instructor in the .election center, certified .elections, registration .administrator, the Sierra .program at Auburn. Before .Auburn, she was a voting rights .researcher at the Brennan Center.For Justice..While completing her graduate .students studies at Kent State .University, Bridgett coordinated.the McNair scholars program, .which aims to boost the number .of low income, first generation,.and under represented college .students who pursue doctorate .degrees. That sounds very cool,.Bridgett. Please welcome the .panel. .[APPLAUSE] . All right. Rey, shall.we start with you?. >> Sure. It's always .hard to be the last panel right .after cookies. So I know we'll .still try to keep you up and .awake. .Before I start, I just have a .question for the audience. I .myself appreciate and it's an .honor to be here because -- and .you'll see me taking notes. .It's not for rebuttal but .because I'm learning quite a bit.myself..I'm very interested in /SKWREPB ./TPER and the risk limiting. . Part of it is one of .the things we do learn and what .we have learned, if anybody in .the audience .knows what R&D stands for, .they're probably wrong. It's .not research and development. .It's rob and duplicate. In .elections, that's exactly how .all these notes here, what we're.going to take away. I'm going .to show you the wonderful things.we're doing in Maricopa County .as far as provisional balloting .and some of what we say, hey, .that looks familiar because we .probably robbed it from you. . Just so you know, .we'll start out here with the .photo up there is not me. That .is our new recorder. He is that.tall, tall as a city building in.Arizona..And if you haven't met him yet, .I know that he was just up at .training, and if you haven't, .he'll find you. .Part of the process we're .going to talk about is the .provisional ballot close .post-election. Before we get .there, I want to tell you .quickly, briefly our story of .how we got to provisional ballot.tools we're using. Where we .were in 2016, I won't reiterate .what everybody's already -- some.of you have experienced that .four-hour line. That was us in .Maricopa County, 2016, our .presidential preference. We .were in the media..Some of it had to do with bad .data. One of the things I will .point is we've done data .analytics, but you don't update .those analytics and say, hey, .there's a factor there. We'll .call it the Trump factor, if you.will. Most of the time in our .presidential preference, we're .late to the game. It's over by .the time it's in Arizona. . So for lack of a .better word, it's a dog and pony.show..There's already the.heir apparent or somebody .nominated. We didn't adjust our.analytics. The story is where .we were, what we were looking .for, where we are now, and we'll.get to the provisional .balloting, the tools we're .using. .So we have -- and, again, I'll .go through some of these pretty .fast. The presentation's on the.EAC, if you really want all of .the data..Polling places, 724. We have 2..2 million registered voters. We.have 1.6 million of those on our.permanent early voting list. .It's a big, daunting task to .manage that kind of volume..We still do preprinted ballots .in 2016. We had all of these .lines..What we were looking for is .we're trying to create a system .that increased voter access, .engagement, was economical, and .scaleable. We weren't just .looking for a unicorn, we were .looking for a golden unicorn. .Those are really rare. .Again, we wanted that..Not to.some of the e-poll books. For a.county our size, we needed .something different. We needed .something that would allow us to.do this on a realtime basis. We.were having, as an example, when.we get to the provisionals, we .were upwards of 100,000 .provisionals in Maricopa County .for 2012..2016, we it better. We had .56,000 provisionals, and those .are things that are worked after.election.and long hours of processing us..Again, doing a better job on the.front end to get people to .update their practices we're .focusing on, but that's .something we need to mitigate. . What we did is took an.approach saying, we have 15 .counties in .Arizona, Maricopa is the largest.-- the other 14 counties call us.the great state of Maricopa .because we make up the .majority, and we have quite a .bit of a robust IT staff. Not .that every county has that .luxury, but what we've done and .we challenge our IT staff to say.give us a better process, a .better check-in than what we .currently have even in our .e-poll book environment. . What they did is they .created this check-in process .that ties directly to similar to.what was presented earlier in .the ballot on demand .environment, but .it is robbing and /TKUP /* .duplicating from an industry. .If you flew here today, you .probably didn't talk to an .agent. You did a precheck. And.even some of our mailing of our .early balance /* ballots, Orange.County we stole from them with .their rely a vote. Thank you .very much, Orange County..Years ago, we were using a .system from .pit any bows.for a credit card check. . Instead of having one .e-poll book and a clerk behind .here or two, as we did in the .presidential preference, which .had long lines, we can set up .three to six terminals that are .precheck-in..You basically are coming in..The system is set up with a bar .code that can scan driver's .license. You basically .interface. There's privacy .screens. This is a model here. .Mr. Commissioner hits. I made .him about 10, 20 years younger .here. So he doesn't get that .too old card. I didn't want him.to get that. So an example. .This is what you would see as a .voter. You precheck-in. And we.have a gatekeeper for the people.-- is anybody from Arizona here?.Good, just me. . But nationally, if .you're watching, we do have a .gatekeeper. We do still require.checking IDs at the polling .place. It's part of our state .law. Commissioner Hicks would .come in, is this your name?.And 95 percent, our data shows .-- and even from our e-poll .books. 95 percent of our voters.swipe with a driver's license. .Those voters walk right in. .They swipe. Is this your .address? This is how we get to .our story about provisionals. .For those percentage, 26,000 of .our .56,000 provisionals, 2016, were .address changes. That's all .they were. But they were .three-part form, paper form, .that -- and excuse me, Ethel. .And I use that name lovingly. .But Ethel, who's usually 80 .years old, is filling this out, .and it takes time to spell .Valenzuela, and it's usually not.right..There's a point of where.this is where we eliminate a lot.of provisional. .A fillable form basically. With.our SiteBook technology, we're .not going through a third party .vendor. We're not coming .through cloud. We're going .right into the voter .registration database. I will .tell you the party is left blank.purposely for Commissioner .Hicks..It use the /*ed to say party. .And a place to sign. I'm going .to go through really fast. I've.only got a couple of minutes to .get to the provisional part..Basically, the voter checks in. .They precheck in. The ballot is.automatically set that's correct.for him. We have 7,000 styles .in our upcoming election .primary. It doesn't matter. .They can go through here, and .they step over here to wait for .their name to be called. It's .the Starbucks of voting..I was going to put a Starbucks .logo but copyright infringement .and such. . You place your order, .step to the side. And .check-ins, just so you can get .here, as far as the data from .the SiteBooks. We can actually .get down to minute by minute .check-ins, and it's not going .through a cloud, not going .through third party..We're getting actual.live open and close data, as far.as seen on our dashboard. Eer .even using it for board worker .payroll. You can see Benny here.worked 15.3 hours. One of our .hardest workers. But it's a .tool that is not only sped .up check-ins, in our 2017 .election, local election, it was.a small election, .only a million voters eligible .for that election. We call it a.baby election. It took an .average of 31 seconds to check .in and only 44 provisionals for .that election. Similar type .elections, about 3,000. So .that's a drastic 98 percent .reduction. . 2018, we had a .congressional district special .election. Congressman Franks .opted to resign for some reason..We'll leave it at that. And we .had to have a special election..So in that, we had actually 531 .provisionals..Similar type election would have.been about 52,000. Because .we're updating registration .right on the back. We got all .these analytics that allowed us .to then look at these .provisionals..So we can now grab data specific.to .provisionals and say.even if we get the election and .we're at 9,000, 10,000, we're .currently looking at the data .and saying where are the high .attrition rates? Where are more.provisionals being cast? We can.look at the data big time and .say we have areas that are .apartments around gateway .testing grounds, which they have.high transient employees. So .those people tend to move a lot..So we know we can focus there .with this data. .Last but not least, we can also .know then we can increase board .workers in that area. We can .have options for three, six, or .12 precheck-in. So basically .our provisional tools allow us, .no matter what with this .particular option of where we .were, how we got here, to do a .lot of things. We'll save a .couple things for questions and .answers, but the one big .thing is on the provisional .tool, this SiteBook has given us.the ability to do realtime .processing, but more importantly.to do it on the fly. By that I .mean, for those of you that do .realtime provisionals, a whole .two years ago for us, you had to.wait until after election day or.election night..We're actually the /* getting .that provisional. The minute .the voter casts, they say .they're not registered. They do.an address change, they're .fixing it right then and there. .But not registered because they .said .their name is Rey-r-e-y, and .we've got him as Reynaldo. .We're getting that data, working.it, and dispositioning it. We .don't have election night .provisionals any longer other .than the voter that votes at .6:59 provisionally. We've .closed our polls. At 7:01 the .data is there. 7:05 I think was.our latest..And we have our provisionals .done by 7:10 p.m. that night. . Again, some good tools.to look at. Hopefully, you .know, we are looking to program .our SiteBook..We actually are -- my recorder .is in NACO today receiving an .award for that SiteBook, and .he's looking to open source .that..Not, again, any e-poll book .vendors out there, to try to .take any business. When we .perfect it, know that as a rob .and duplicate, as penance for so.much we've stolen from other .counties, we're willing to give .back. So that's going to be an .option if and .when we work out any and all .bugs. That's our provisional .tool we're using and data we're .capturing so far. . >> We're going to talk.about tools a little as well in .my presentation. First of all, .it's an honor to be here. I .want to thank the EAC for .inviting me. About ten weeks .ago, I took off my hat as an .election official and partnered .with the democracy fund in what .is turning out to be a really .exciting project, looking at .ways we can increase .trust in elections through .audits, standards, and testing. . So the focus of my .thoughts .this afternoon when we think .about post-election data, we .think about the way we evaluate .our elections, the way that we .audit our elections, I want to .think about it through the lens .of increasing trust, and I've .included this .statement from democracy fund's .trust in election project and .that our focus should be .creating a system that is voter .centric and efficient and .inspires trust in our .electorate. .Let's talk about post-election .data for just a minute..We live in this era where data .is almost ambient. It's .everywhere. We live and breathe.and work with data flowing all .around us. Nothing -- there's .never been, I think, a bet /*R .time in elections to be in a .place where we start to be data .centric and data driven..So how do we do that, and how do.we capture that?.First we start by asking .questions. .Some of the things we can think .about after an election is who .are our voters, and how do they .behave? There's some counties .that have done an excellent job .post-election of capturing this .and showing this. I'm not going.to talk to all of these. I .really just want to illustrate .the .amount -- the data sources that .are out there and the ways we .can think about framing .questions around that. I will .talk about that last one on .there. That one has been really.interesting. Geo-mapping is .something that's growing. I .know we'll see more of that as .we start to go through the .precinct thing, .but we used that in Arapaho .County as we tried to plan for .vote centers to look at .geographically where voters were.located in contrast with that .voting location they chose to .use. . Most of the time, it's.the closest location to them but.not always. That's really .hopefully when you're trying to .plan. How do your voters .behave?.How well do you communicate with.your voters?.There's a /PHREGT /RA of data .after an election that you can .look at to look at whether the .instructions were understood. .The instructions for completing .a ballot, returning a mail .ballot, the information that you.put on your website, the calls .that come in can be really .revealing. Amber talked about .that earlier in changing .processes to better inform .voters. . One of those that's .new to me .that I think is exciting is .states and .jurisdiction that's are .purchasing new voting equipment..Quite often now there's an .electronic adjudication portion,.and there's all kinds of data .embedded in that. One of those .things is ambiguous marks or .marginal marks. And we really .have the opportunity now to go .in and look closer at that and .how often they're made and how .they're made and how that ballot.was interpreted because of that .and maybe come up with better .instructions or better design on.our ballot..So that's an exciting thing in .post-election data. .How will our processes -- how .effective are they? How .effective is our training? .There's a lot of things we can .look at when we try to answer .that question. I think one of .the most effective in the county.I just came from was looking at .the poll worker hot line or .troubleshooting logs. We did a .really thorough evaluation after.2016. One of the things that we.found was a series of questions .that was being called and .repeated over and over again. .Something that was covered .repeatedly in training, but .obviously our training wasn't as.effective as it should have been.and we weren't getting that .message through. So we weren't .back from square one, and we .rebuilt our training based on .the calls that were coming in .and the troubleshooting logs .that we saw. .One of the things that I put on .there that I think might be .interesting is very low tech. .So we're not talking about a lot.of .data sets or spreadsheets is .inventorying of ballots and .supplies. In my prior job in .Utah, one interesting thing we .did after the election was to .inventory all the supplies that .came back. And for about three .or four elections .in ray /* a row, we noticed on .our inventory sheets this .listing of pencils. The only .thing that was interesting was .we didn't send pencils out..So we did a drill down into the .root .cause, and what we found is we .really needed to revamp our .election form. Poll workers .were concerned about filling out.the form correctly, so we were .bringing pencils from home so .they could do the tallying and .then erase the numbers and fill .it in in pen. . How effective was your.planning and your resource .allocation? We've heard a .number of individuals today talk.about the MIT resource .allocation tool, the line .management tool, and that's just.as effective when you're .planning for an election as .doing a post-election evaluation.or audit. It gives you an .opportunity now to .plug in real data and see how .that compared with what you .planned for in .terms of.how long did voters have to .wait. And what your throughput .was. I can see I'm running out .of time. So I'll skip through .that. This is what we want to .talk about. How ballots were .handled, and were they tabulated.correctly?.We have our ballot .reconciliation logs, our ballot .manifest, cast vote record, .adjudication audit logs, and .custody reports. I know there .are election officials sitting .out there, and those that are .listening are probably thinking,.well, she captured a few of the .data elements we might look at .to do some sort of audit or test.or review, but she hasn't .captured all of those, and .you're correct..There is a lot of information .out there, and because there's .so much, you would think that .we, as an election body, would .be doing .that and capture these regular .robust .audits and regular reviews, and .all the jurisdictions in the .nation would have the world's .best elections. .I want to just pause for a .minute, though, when we think .about why and what makes it .difficult to adopt those..I want to talk about surgery and.the history of surgery..So in 1846 William Morton .administered ether for the first.time. It's the first time that .anesthesia was used in general .surgery. I think you can see .right away why it was a win for .both the patient and the doctor..And it was quickly adopted. .Within two months most of the .capitals .in Europe were doing it, and .within six years, it was .something we saw done in just .about every hospital in the .country. Why is that?.It was very visible and .immediate effect, and folks .could recognize that. And we .see that, right? In the .election community. We see .processes or ideas such as vote .centers where there's a win for .the voter, there's a win for the.election administrator. It's .very visible in the media, and .so we adopt that. . Post-election audits .and risk .limiting audits and reviews are .more like germ theory..So in 1877 Joseph Lister .performed surgery and wanted to .prove his theory about .antiseptic surgery and using .carbolic acid to reduce .infection, and .he did that and reduced it by 85.percent. Mortality rates for .those that had the process done .went down considerably. But in .over a generation, less than .half the surgeons were using .that. Why do you think that is?.It wasn't visible, and it wasn't./PHRAOED. Weeks would go by .after the surgery, and they'd .soon moved on to the other .patients, or in our case, other .elections, and we didn't need to.think about it so much..How do we lessin the pain now .and .highlight the long-term benefits.of robust audits and testing?. I will tell you what .the answer is. An election .official in a very small .jurisdiction where elections is .one piece of what you do and the.other months out of the year, .you're issuing business licenses.and you're doing .planning and zoning and you .might be doing motor vehicle .licenses..So I think we need plug and play.tools .that we can.give to officials to better use .the data and do so in a way .that's efficient for them. What.does that look like? We need .plain language definitions. We .need to clearly outline the .necessary steps and deadlines, .and we need to calculate the .workload or have a calculator to.determine the workload so they .know up front what it's going to.look like. It involves .checklists, tools, templates, a .way to practice and to .collaborate with one another. .So I'm going to walk you very .quickly through those of you .thought in eight to ten minutes .I was going to tell you how to .do a successful statewide risk .limiting audit, I'm afraid I'm .going to disappoint you. I'm .going to quickly walk through .what such a playbook might look .like, what such a checklist .might look like, and just let .you know there are those .collaborating to put some tools .like this together. .And a lot of these ideas I .borrowed from folks that have .been doing audits for a very .long time. Those are .individuals in the private .sector who have been doing .financial audits. I won't read .through this. Essentially, .they're saying the same thing. .You've got to create .definitions. You've got to .create specifications..You've got to have the questions.available so that this process .becomes very routine. . When we talk about .risk limiting audits from the .perspective of an election .administrator -- so I'm not .going to talk about software .tools that are necessary..I'm not going to talk about the .responsibilities of the state to.carry out something like this on.a statewide level, but I just .want to give you a quick .overview of what it might look .like for an election official. .Most of these things I think .you'll recognize you're doing .already, and most of these .things are things that you're .already familiar with. .So it might include a workbook .that talks about prescanning, .how to organize and track your .ballots, creating a documented .plan for doing that, for .reconciling your ballots. .Thinking about ballot batch size.and what that means and how .that's going to .allow you to ensure that you're .not going to lose or .gain any ballot s throughout the.process. Ballot storage .containers are the most simple .but most essential part of the .whole audit process. . Think about this. .What we're saying is that for an.election with 400,000 ballots, .we're going to be able to .pinpoint to the exact ballot to .pull that for audit. So .thinking about containers, .labelling those, creating unique.IDs for those, and thinking .about a thoughtful storage plan .for those is essential. .The other key element in your .preplanning or pre-scanning is .your ballot manifest. Some sort.of a tool to map where all .of those ballots are stored and .how. .We might also in such a playbook.talk about ballot scanning. .That's something you do every .day, whether you only scan on .election day or scan throughout .the early voting process or .post-election, there are some .really simple steps that you .have to go through. The key one.is reconciling and verifying the.number of ballot scan matches .that tracking form you're .working with every day. I think.you can see where I'm going with.this. Organization really is .key. .So we talked about risk .limiting. We talked about the .theory. We talked about the .statistics behind it. But as .far as an election official is .concerned, the key to a .successful audit is the .organization process. .Let me skip quickly and talk .about ballot manifest because I .mentioned the need to have very .clear definitions. There are a .variety of academic definitions .on what a ballot manifest is, .and I think this is essential .for every element within the .risk limiting audit process is .thinking what's the simplest way.that I can define this that's .meaningful..For me it means creating a map .or a legend with how individual .ballots and batches of ballots .are stored..It can be an Excel spreadsheet, .a paper log, or an application .developed by a software .developer. But it has to be .independent of the voting .system, and it has to be .formatted in a way that shows .certain fields that are going to.be tied to your cast vote . So the device the .ballot was scanned on, the .unique batch number, the number .of ballots that were scanned and.the storage container ID. .It might look like this. I've .got two examples here from two .different counties in Colorado. .This is for a statewide audit. .If you're doing this countywide,.those county names might be the .names of your individual .precincts. But the elements .there are actually really pretty.simple. It's the device. It's .the batch. You can see they're .sequential. The number of .ballots scanned into that batch,.and then the location. I've .seen counties do something as .simple for the location as just .a box. I just visited a county .that had six boxes full of .ballots..So it was box 1, 2, 3, 4, 5, and.6.. We talked about ballot.storage. I'll skip that one. .There's some essential steps .there .that are critical to a .successful audit. .And then reconciling each day. .So, again, we have these data .sources that are available to .us. We have our cast vote .record. We have our ballot .manifest, and we have the .information from our voter .database or our poll book so we .can assure those totals are .matching before we move into the.audit process. It's really .critical versus waiting .until the end of the election to.resolve discrepancies that daily.those get researched and .resolved. And this is what a .cast vote record would look .like, and I think that's, .again, something that may be on .its face can look a little bit .daunting, but really it's .simple..It's going to be unique to each .voting system vendor, but a .successful audit .means understanding what each of.those fields are and what they .represent and how they're going .to tie all of those pieces .together when you go to .reconcile or audit your ballots.. So you can see here .each individual ballot is .assigned a unique .number and that unique number is.then identifies which tabulator .or scanner it was scanned on, .which batch it contained, so in .the example I've blown .up there, you can see ballot .44851 was scanned on tabulator .3, and it was the 97th batch .scanned on that scanner. And .within that 97th batch, it was .the 70th ballot. So it's pretty.simple actually. And there's .some other information there. . The audit itself, at .that point, you've done all of .the hard work, and now it's just.about retrieving the ballots .using the system that you've set.up in entering the voter .markings to do a comparison. . Easy, right?.We all are ready to jump in and .do that..And I realize that I've left -- .like I said, it's a really .complex process, and it's not .something that we're going to be.able to communicate in eight to .ten minutes..But I want you to think about .audits, risk limiting audits .specifically, but other types of.audits in thinking about a path..It's not an immediate .destination. Sometimes we think.I don't have the resources. I .don't have the infrastructure. .The laws in my state or .jurisdiction are not set in a .way that would allow me to do .this, but if you break it apart .and recognize there are .different places along the path,.there are a lot of things that .are low tech and things that ./TKOUBG right now to put .yourself further along that path.that are really .fairly simple and easy to fairy .out. .you can do right now to put .yourself further along that path.that are really fairly simple .and easy to fairy out. .I know we talked about .definitions, and I was asked to .delve .into these a /HREULT /* little .bit more. We're part way there..We were able to come up with .complex theory and come up with .simple definitions. I've given .you three here..Batch-level comparison audits, .ballot-level comparison audits, .and ballot polling audits, and .think about a way to simplify .that even more in a way that we .can see it and understand it. .So just in conclusion, how do .we use post-election data to .focus on prevention rather than .intervention? Risk limiting .audits are a way to focus on .intervention and realize you .have a problem with your voting .equipment and how do we think .beyond that? Why not only audit.and test our voting equipment? .Why not validate other critical .components of the election .system as well? I'll skip that .because of time. . So I am focused right .now on creating a collaborative .state and local election .professionals and subject matter.experts to do just that, to .think about different points .along the system, different data.elements that we can look .at pre and post-election and .provide tools and guidelines and.best practices and assistance .for jurisdictions to implement .those..I think we all need to become .audit experts..A lot of times we turn to .professionals, advocacy groups, .the /*. But as election .officials, we need to explain .how these terms work, how these .processes work so we can be .driving that policy in our own .communities..Charles talked earlier about .experts. We're starting to see .a number of experts. We have .legal experts, communication and.social media experts, data .analysts and IT experts. I .think we need auditing and .quality control experts. But .until then, because I understand.the resource limitations, we .need to learn to be pit crews .and not Cowboys and learn to, as.Rey said, steal from one .another, collaborate with one .another, and that's something .that's in the works. I don't .have time to talk about it, .but a shared customizable tool .to help .expedite the risk-limiting .audit, the actual comparison .process.. And I'll just conclude.with this, statement on the .national commission on election .reform. Americans can and .should expect their electoral .system to be a source of .national pride and a model to .all the world. We're at an .exceptional time. I think that .is happening, and I'm excited to.be a part of that. . If you're interested .in collaborating or sharing your.expertise, you know where to .find me.. >> Thanks, Jennifer.. [APPLAUSE] . Next, Kara. Thank .you.. >> Thank you for .having me. My name is Kara .Rahn..I'm the./P*UGSD.director of elections for .Chester County, Pennsylvania. I.was first introduced to the idea.of collecting data from Charles .a few years ago when I .participated in a Department of .State presentation..I felt, at that time, somewhat .out of control of all the things.that an election director has to.manage. .So what I chose to do in that .time and was inspired to.start to collect the data so I .could be in control of some of .the factors that affected my .staff. And data is the place .that you can find that control .and that confidence and that .comfort. .So for me, it was a reaction to.2016, which a lot of us during .the course of today have fallen .on our swords, where we know we .can do better .or with data could be more .efficient. So as an election .director, as a county employee, .I have a balancing act, like .many of us..We have to manage lines, long .lines in .Chester County are a common .conversation with our .commissioners and myself..Meeting the expectations of the .Pennsylvania Election Code. I .asked the question earlier .because I think there are so .many wonderful ideas here, but a.lot of us are trying to tie that.back to how Pennsylvania law .requires that we do certain .things..So there's another piece to .balance. .Balancing county resources. .We're a government agency..We are lean, and we are .obligated to be respectful to .the resources we're given to do .our job and spend them wisely. .And then overall, we have an .obligation towards customer .service. The voters on election.day and other times throughout .the year, of course, .are our customers, and they will.be grading us, so to speak, on .the experience that they have .for the 15 minutes that they're .in casting their vote..So to keep that in mind, we have.a customer service piece to this.as well..So they were some of the drivers.to me .trying to understand the.constraints that we were under .and also still trying to make .those improvements moving .forward. .The other thing that just in .general, I think, a lot of .industries .are facing, and certainly .elections, are .we're at a little bit of a .conflict as .far as the expectation for quick.service..We talk a lot about everything .today is fast with its .technology..If you talk to a child, an hour .waiting for something is .torture..So that is another factor that .comes into play with voting .where we have rules and .regulations set up and equipment.and other things that are not .quite set up to be as responsive.as the expectation of our .voters. . I tease that, if you .ask a .poll worker how did the lines go.today? They'll say, oh, pretty .good. If you ask a voter, one .voter might .say, oh, I had to wait 10 .minutes. Another voter might .say, I had to wait 10 minutes. .So every voter that comes in has.a different perspective, and we .do need to respect those .different perspectives as we go .through these processes and .looking for improvements, but .that's hard because those .expectations are literally all .over the place. .So just a little bit about .Chester County. We have about .350,000 registered voters, 228 .polling locations..We have some very small .locations. We have some larger .locations. So the logistics of .each of those locations is .pretty unique. We are a paper .ballot county. We have hand .mark paper ballots, and .then we also have a DRE machine .in each precinct as well. .As most of us know, we have .four-year election cycles..So no election matches or .mirrors the one just previous to.it. So a lot of times the data .we collect really has to have .some run time in .order to have good comparisons, .which is .a unique hurdle for election .directors. We're not at Disney..We're not taking daily data .points or hourly data points. .We have two days a year to track.the information, and sometimes .we have to wait a few years to .actually compare so. .So if I can make a plug for .anyone who hasn't jumped into .the data collection game, just .start collecting it. You might .not know what to do with it yet,.but if you miss the opportunity .to collect, you're just .prolonging that .cycle from actually the /TKPWE.getting started, which I was .equally guilty of up until about.a year and a half ago. . And then as you know, .voter turnout varies..This past election, we had a 16 .percent turnout, which is a .bummer, but that's what we had. .So to compare that turnout to a .presidential would be silly as .far as resources and .efficiencies. But the .expectations of the voters are .the same, whether they show up .every .election or only every four .years. .And just to share there, we have.the equivalent of about 1,200 .full-time, full day poll .workers. We average about the .equivalent of .about five full day staff per .precinct in order to keep the .show going on election day. . So I mentioned we .started with .the algorithm provided by the .Cal tech MIT research and found .it extremely easy to use..We joked a little bit about my .nuclear background, and I was a .process improvement person .there. So data was everything..Again, I went back to what I was.comfortable with. With data, .you can understand the situation.and spend your resources wisely .on how to get after that. . The other thing I like.to joke is nothing can go wrong .in a nuclear power plant, and .nothing should go wrong on .election day either. The .pressure is there..So what we do with the data is .we try .to deduct where we should be .using our resources. We can use.our resources most efficiently .and make some educated forecast.. So I just have a .couple samples here to kind of .show from where from a volume .perspective it may not be a big .deal, but from a customer .perspective, it hurts..So in this example, this last .November, on the left here, I .show total number of voters that.showed up that day, and how many.lines they were running. We do .not have electronic poll books. .We have paper poll books in .Chester County..So our bottleneck is at the .check-in table.. One thing that we did .through some anecdotal polling .is we realized the number of .privacy booths and the like we .started providing clipboards .even in some of our polling .locations .that, if someone really needed .to get .out.and were less concerned about .having a privacy booth to stand .in, we offered a clipboard to .let those get out quick letter .if they were in a rush, and .those that wanted to wait for a .privacy booth to open up, we had.that option as well. So we just.provided simple things like that.to get folks moving. Now, I can.do that. I have paper ballots .in that example. So my line .focus is really at that check-in.table. And I call my poll .worker Sally..I know we have an Ethel and .Dimple was another name. . So here is an example .in this polling location. We .only had a 30 percent turnout. .We asked poll workers to track .at the top of every hour how .many folks are standing in line .and what line they're in. Very .simple and easy for the poll .workers to keep track of. They .set a little alarm on their .phone, jot the number down. .That's all we need. Quickly, I .could say, to the judge of .election ins this particular .precinct, you need to run more .lines in the morning. But the .rest of the day, you could .probably settle down. So if you.only have so many resources, .where are we going to ask them .to double down? It's going to .be in the morning. .Now, that particular poll .worker, as an example, if you .walk in .and you go to vote and your last.name starts with a \"B,\" you're .really annoyed there's no one .else in that other line, but you.have to stand in a line of ten .people waiting to get your .ballot checked in..That's something a poll worker .might not consider as something .they need to worry about because.they're busy and .doing their job, but from a .customer satisfaction .perspective, that seems like an .easy one to get after. That's .something we'd share with that .judge of elections that as low .as a 30 percent turnout, we .already have a problem. What .are we going to do at a 40, 50, .60 percent turnout?.So instead of having a .conversation that sounds like .the election director is telling.the elected .poll worker how to do something .differently, we're using data to.normalize that conversation and .then together come up with .solutions that we think might .work .for this particular precinct. .And this goes for any other .business where you stand in line.and you're annoyed there's 15 .closed checkouts and only two .people working and you're .standing there in line. It's a .similar feeling that people have.when they walk into a polling .location. . This one, not a large .turnout, 395 folks all day, 16 .percent turnout. They had a .small line all day..One more person would have .satisfied no one having to wait .in line at the day. In a 16 .percent turnout, I wouldn't .spend a lot of time trying to .fix this precinct. But in a .presidential or gubernatorial .election, I'm going to make sure.this .judge of elections isn't running.the precinct the same way they .always do because this is not .going to work for a higher .turnout. Again, the data allows.me to have the conversation in a.less threatening way since we .all here in Pennsylvania have .elected poll workers to try to .manage or work with. .Then what we do too, if the poll.workers are able to or have the .time in our numbered list of .voters book, which is another .book our poll workers are .required to use, to track at the.top of each hour, just right .next to the person's name, who .is in line at that hour so that .we actually can then go further .to see how many people voted in .that particular hour just by .taking a quick look at our .numbered list of voters book. .So that's another way for us to .see if the line's consistent all.day or if it just so happened to.be at the top of that hour there.was a line..These are simple things we can .ask our poll workers to do on .the spot versus putting out the .other fires like tornadoes and .the like back at the office. .So we do not, I mentioned, have .e-poll books. We're looking for.something else like .that to take us to the next .level within .the polling location as we look .for new equipment. But I think .that it's important for us to .realize that the volume is .likely not going to change .coming into the polls. Where .the line builds is what's going .to change. So we need to .consider if we fix kind .of that whack-a-mole game, if we.fix one, it will likely create .an issue or a line somewhere .else. So I'm really trying to .take the data .that I'm using now even to go .into what type of new election .equipment I'm going to purchase .so that I'm really looking at .the big picture and not just the.lines that are forming at the .front desk .because that will likely .dissipate or .hopefully not become an issue at.all if we select the technology .to help us along there. . So the other thing is .that we've talked a little bit .about is our poll workers. .There's a learning curve. .Anything new can be scary .sometimes. But what we've .decided to do is build .in the data collection as part .of their poll worker training as.if it's just another thing .that's expected to be done at .the polls that day..What we did in the beginning was.we asked for it as an add-on, .and we got a very low response .rate from our poll workers..The new poll workers are coming .in, and we're just kind of .making it part of our process, .and we're up over 50 percent now.on our return rate. . The other thing we're .learning is making sure the way .that we're training to collect .the data is important. Bad data.is not going to get us anywhere..We've had some interesting .responses to our form that .didn't quite match what we were .looking for. So we're still .working through it. We have .absolutely not arrived, but .we're trying, and I feel .confident already with the data .that we do have, the pockets of .the polling locations .that we maybe need to put some .more resources to.. I think bottom line, .even with the data collection, .in Pennsylvania, our poll .workers are still the most .critical piece to the puzzle. .They need to be educated..They need to be willing and .interested, as I am towards this.data and towards that customer .experience and customer .satisfaction. What happens on .election day can make .or break the reputation of the .county, frankly, of someone's .experience. . So what we're doing, .just to show an example of what .we're doing with the data .besides just coaching our judges.of elections on what we could do.better is we're now using the .data to look at where the .pockets are of where we're .having a hard time finding poll .workers..To take the show on the road and.go to businesses who offer -- .maybe they offer .days off if you do a United Way .community day. Maybe they can .add poll workers to the list of .opportunities that their .employees could have the day off.if they're going to go offer .some civic support in the polls..It's a nonpartisan effort. It's.VVSG their community. Why .aren't we looking at VVSG our .polling locations like we do .other nonprofit opportunities .throughout the year?. We go to rotary clubs..We'll talk to college students, .high school students..We have some very active honor .society groups that will send us.student poll workers, and .they're amazing, and they're a .really nice balance to the .generational mix that we have in.our polling locations now.. And I think we're also.going to need to identify which .polling locations this data is .requiring us to .do more of a hands-on survey or .field review versus just .collecting the data from the .poll workers to get after what .they might need specifically..We talk a little bit about the .characteristic of each precinct..We have somewhere the nursing .home bus loads up and goes to .vote at 10:00 a.m. So that .particular location has a .different set of needs and .expectations and different time .of day than a heavy commuter .area where the rush is going to .build at 7:00 a.m..So to treat everyone the same .is, I think, a waste of county .resources. We really need to .dig in to get the specificity .that the data can provide us and.also the comfort to know that .certain precincts are doing just.fine, and we have the data to .feel comfort in that and not .just the anecdotes that we .might hear from our poll workers.or from others. .Okay. That's it. Thank you. .[APPLAUSE]. >> Ready?. >> As I'm going to be..So in many ways, it's .appropriate that I'm going last .because what I hope to do in .this presentation is talk about .the role of evaluation and leave.you with some thoughts about how.you might approach that while .integrating some of the things .people have talked about .throughout the day with respect .to the .way they have used innovation to.address the challenges they've .experienced in their .jurisdictions and also try not .to break the clicker. Okay..Oh, look at that. . So evaluation is the .systemic investigation of the .merit, worth, or significance of.any object. So when we think .about this as it relates to .election administration, .evaluation is really an .opportunity. It's an .opportunity to gather data and .demonstrate the ways in which .policies or programs that have .been enacted in jurisdictions .and can enhance the way you do .your jobs and perhaps more .broadly, the way citizens .experience the democratic .process..And so while we often think of .evaluation in real social .scientific terms, it clearly .doesn't need to be. It can be .-- the nature of evaluation .can take on a variety of forms. .It can be very descriptive just .thinking about gains and .reductions or also be .statistically rigorous. But .regardless of the approach, .evaluation should be something .that isn't intimidating but .something that .you view as a resource or a tool.that can continually be utilized.as we think about ways to .improve and enhance the .administration of elections. . So why do this? If .you haven't been convinced yet, .here's some other reasons..So one reason to engage in .evaluation is it allows you to .step -- am I too close to this? .It allows you to establish model.programs and best practices by .providing feedback about what .worked and what didn't work. So.if you can think about some of .the voter outreach and .information approaches people .have discussed, poll .worker training, also the way in.which equipment is delivered to .polling sites, all of these .things are opportunities to .consider places for improvement..If we think about public .administration more broadly, .evaluation .is a tool of good management and.quality improvement..It can be used to gain insight .into effective strategies on how.to improve performance, measure .impact, improve implementation .and effectiveness, better manage.limited resources. One theme .that's consistently been .discussed throughout the day is .the best way to utilize limited .resource that's people who work .in elections have. . It's also an .opportunity for you to document .program accomplishments, to .justify current program funding,.to perhaps convince the powers .that be that you need a little .more money..And also, evaluation is an .opportunity to satisfy ethical .responsibility to the public and.demonstrate the positive and .negative effects of things you .might be doing in your office. . So this is .particularly true if we think .about the ways in which .society is increasingly .concerned about accountability .and results..Evaluation is the opportunity to.demonstrate what you did, why .you did it, and how it benefited.the public. And beyond that is .an opportunity for you to .contribute to the body of .knowledge that exists in the .area of election administration,.to share with your peers what it.is that you're doing, .the successes that you had, and .allow them the information so .they can think about how they .might integrate what works for .you in their particular .jurisdiction. .And so just like we talked about.there being a variety of ways to.address challenges, there are .also a .variety of approaches to think .about what sort of solutions are.going to be in the best interest.for your particular office on .the front end. So one approach .to doing this is a logic model..And so a logic model is .basically just a map..It is a simplified picture of a .program initiative or .intervention that is responsive .to a given situation. It .requires you to think about the .resources you need to prioritize.the problem or challenge you're .currently facing, and perhaps --.not most importantly, but also .importantly, it .requires you to think about the .short .term, intermediate, and long .term outcomes of the .intervention you implement, and .those things are often .measurable, right?. So it allows you to .think about what it is that you .need to do in order to address .your specific issue..And so a situation is simply .just a challenge. So if we .think about the beginning of the.logic model. It's a challenge .that you're faced with, whether .it be the result of a particular.problem you've experienced in a .previous election, or if you're .just thinking about ways to .enhance what voters experience .on election day. You would then.think about sort of what .resources you have to allocate .to address to this particular .problem, what it is that you're .going to do, and then .again, like I said earlier, what.the .outcomes are going to be once .you implement whatever policy or.practice you've decided to use. . Here is -- oh, see, .okay..Here is an election .administration specific logic .model introduced by two of my .colleagues, Dr..Kathleen hail and brown. You .can see to the far left that the.issue discussed in this .particular jurisdiction is slow .voter turnout..And then below that, you see the.resources that are going to be .necessary to address this .particular problem. And then .immediately to the right of the .problem, you see the activity .that, in this particular .scenario, they decided to focus .on, which is a nonpartisan .reminder that is sent before the.election. . If you consider .short-term outcomes, obviously .one thing is the voter still .shows up to vote. If we think .beyond that, you think about the.person becoming a consistent .voter, so voting regularly in .elections, and then more .broadly, you can think about .increased democratic functioning.as we think about participation .being an important component of .living in a representative .democracy. .So in thinking about the types .of evaluation, there are many, .and here are just four..So the first is the formative or.planning phase. This might be .where you would .introduce a logic model to .address a specific issue..But the idea is that you during .this phase will think about what.is actually feasible, .appropriate, or acceptable..So that clearly is going to vary.from one jurisdiction to the .next because we all face .different challenges or .opportunities in different .environments, whether that be .related to our resources, the .political environment, et .cetera..But the beauty of this .particular .phase evaluation is that it .shows you or should help you .understand the extent to .which the program is going to be.needed or understood. And it's .useful for making modifications .before you make a major .investment into any approach to .address a particular problem.. For the next type of .evaluation is process .evaluation..And so this is the type of .evaluation .you would engage in once you .have begun a program and .throughout it. So it's .basically monitoring..So if we think about the .situation in Colorado or in .other states where we look at .peaks and valleys and voter .turnout and think about the ways.in .which we might better allocate .resources, that's an example of .process evaluation. So the .beauty of process evaluation is .it lets you see over the length .of your .program how well it's working .and can also alert you to any .challenges with respect to the .protocol that you might need to .address to ensure it is as .successful as you anticipate it .to be to address the challenge .that you initially began with. . There's also outcome .effectiveness evaluation, which .is kind of self-explanatory. .But it basically allows you to .determine or assess the extent .to which.you've been effective and allows.you to engage in that particular.process as well. . And then lastly, you .have impact evaluation, which .basically is there to just allow.you to determine if .you are -- if you did, in fact, .achieve your ultimate goal, and .it's also useful because it will.allow you to provide evidence or.data to think about whether or .not you're going to proceed with.the program in the future to .allow you to assess the extent .to which what you did was .efficient and effective and also.allow you to think about if .perhaps, when you do it in the .future, or if you do it in the .future, if you need to increase .the scale at which you approach .this particular activity because.it was .just so amazing. . Thank you..So in thinking about the .different approaches to .evaluation, one might ask .themselves which approach is .right? And the answer is it .depends..This is like being in one of my .classes. There are no answers. .But there are questions you can .ask .yourselves to get a sense of .what type of evaluation is going.to be the most useful given the .phase of your project. So one .question is to ask yourself .about utility. So who is going .to need the information from .this evaluation, and what is it .specifically do they need? You .would also ask yourself how .feasible is it in /*? How much .money, time, or effort could you.put into the evaluation? You .also need to think about .proprietary, who needs to be .involved, and the evaluation to .be ethical, and then lastly what.design leads to accurate .information. . So thinking about what.you just said earlier, about all.data isn't good, you need good .data to do good evaluation, to .make good decisions. So the .idea about evaluation is .effectively to be thoughtful and.sort of think sooner rather than.later about what it is that .you're doing and what kind of .questions you would like to be .able to answer when you're done .so that you can identify and .think about and .collect data that will allow you.to complete the kind of .evaluation that is most useful .and add value to the work that .you do..Oh, and thank you.. >> Thank you, .Bridgett..Great. Well, thank you to all .of the panel. I have a couple .questions. Do we still have .enough time to ask some .questions? Great. . I'll start with Rey..Rey, when we were talking on the.phone before -- in planning for .this, you described your .check-in system as an e-poll .book on steroids. And since my .first job in this field .was as a poll worker, particular.liqueurius about how the poll .worker .training went both from the .start to the finish. Like when .you announced you were having .this e-poll book on steroids, .and .what the process was, and how .the poll workers handled it..>> Just to let you know that I .took a tour of Philly, and they .said that 9 out of 10 times, the.answer is Ben Franklin or .cheesesteak. But that doesn't .answer that. .Truthfully, it was actually a .robust -- we have re-implemented.what we call deputy registrar .program. In Arizona you have to.be a deputy .registrar 19-aught to register .somebody. Reactually reinstated.it, not because of registration,.but to train up folks and give .them a certification because .it's still in the statute. And .part of it was grabbing all of .our poll workers and saying we .want you to be experts in the .process, so come back 100 at a .time on every Saturday -- and I .mean this in the tone that I'm .saying. Every Saturday I was .there prior to our launch. .And we launched this -- we .told our IT, just as a point of .reference, 2017, April 1st, .2017. They thought it was an .April fool's joke we wanted it .launched by November of 2017, .within a few months..Bill Gates didn't think we could.do it. Bill Gates is a .councilman in Arizona, not the .real Bill Gates. Probably the .real Bill Gates thought we .couldn't do it as well. .We brought in our poll workers .100 at a time on weekends, .Saturdays, Sundays, and what you.saw on the screen was our sand .box..Was a demo board worker polling .place setup..And bringing them in, we wanted .them to touch this precheck-in. .You're now in the back of that, .a man walking, waiting for .somebody to raise their hand and.say little blinking light, show .me your ID. That's basically .TSA. You can precheck-in, get .your boarding pass. You don't .get the ballot until you show .your ID. That's all they were .doing. Unless they needed .assistance, 99 percent of the .people. .So we walked them through. One .of the key things they loved .about us, because it's somewhat .technical .equipment, we have launched.a T-TECH program where we set up.the polling places. The board .workers, all they need to do is .show up with their inspector and.a backup inspector that has a .badge that launches those .SiteBooks, opens up the payroll .program, which is the most .important thing for them to .check in, .and they don't do anything on .election .night other than secure the .ballots and bring in the edge .cartridge, so to speak, and .their touch screen, and they .don't even touch those. . Part of the training .was have them go through the .environment, and kind of like .Dimple, who maybe was a little .hesitant, the first time thought.you can't do this. We walked .them through. Actually lined .them up outside the corridor, .100 at a time, and said we're .going to walk you through and .have you vote and print an .actual ballot. They looked at .us. On our schedule, it said 15.minutes. They said, we know .this takes an hour. 15 minutes .with that precheck-in process. .And one key thing about that .precheck -in, it goes so fast, .we do get a backlog of people .waiting for the ballot to be .printed. Even that, the one .thing, the feedback we got was .they didn't mind that. They .don't mind waiting five, six .minutes to get their ballot. .They've been checked in. They .feel served. .It's not that they're -- if it's.busy and we've gotten to the .point with those 100 people .coming in, they still -- they .got to comingle with their .neighbors, but not outside .wondering what's going on. .Instead, they know they've been .served and they're just waiting .for that. So part of it with .our training was to this day we .have nothing but 100 percent .positive feedback from both .Ethel and .our step-up teenage program .groups that are helpful in this .technology.. >> That's great. .Thank you..Let's say hypothetically I'm on .the phone with a county election.official, and I need to explain .the difference between an audit .and post-election audits..Can you give me an elevator .speech?.>> What was that? Seven .minutes?.>> He said it was 47 minutes .that he was on the phone with me.the other night..>> One of the most important .things to think about, when you .think .about post-election audits, so .25 states nationally do some .kind of post-election that's .based on a flat formula. We .take 3 percent of ballots or .1,000 .ballots, and it doesn't matter .what the margin is of the races .that are being audited. It's a .flat rate. What's different .about race limiting audits is .we're basing the sample size on .the margins..The closer, the tighter the .margin is makes sense? We want .to audit more ballots. And the .wider is we're going to .decrease. So we gain some .efficiencies there. .The other important element of .that is that we put, we .institute this ./TKWR-D of idea of randomness..So instead of just grabbing the .top batch in the box of ballots .or the first scanner that .happens to be lined up in our .warehouse, we're going to go .through .this process of randomly .selecting those all the way to .the point of the best practice .or the recommendation is to .actually start with a random .seed. I think a lot of you have.seen our dice rolling event in .Colorado..The whole point of that, it .doesn't have to be dice, is that.we eliminate any possibility for.somebody to try to manipulate .the system. So we create a .random seed..We plug that into a.pseudo random number generator, .and we .use that to do a random sampling.of our whole poll of ballots .based on the margins, based on .the diluted margin, meaning the .difference between the winner .and the loser divided by the .total number of ballots that .were cast in that race, and then.we set a risk limit. . So we may say, for .instance, that our risk limit is.going to be 5 percent..This is where I stumble a little.bit, .but what we're saying .essentially is.there is a 5 percent chance that.at the end of this audit we will.not have caught any issues or .something that will have changed.the outcome of the audit. And I.think that particular part, .defining a risk limit, to me has.been the most challenging, and .I've been called to task on that.a number of times..And every time I get to the end,.I get a blank look..So there's got to be a better, .easier way to communicate what a.risk limit is. . I think the important .thing, when we talk about a .traditional audit versus a risk .limiting audit is that we .are basing it on the mar gin of .victory, and we're doing it in a.way that is a random sampling .process. .And then I guess the third .thing is it's not essential. So.it's not absolute we do a ballot.comparison audit. I talked .about that..We're fortunate that states have.the advantage to do that. We're.going to pinpoint an exact .ballot and take a look at that .and evaluate the voter markings .on that compared with the way .the election system tabulated .that. But there are .opportunities within .that risk limiting audit theory .to do what is called a batch .level comparison audit..So we may be in a precinct .count, a .voter facing scanner, precinct .polling place model, where we .may not be able to pinpoint .right to that ballot. .Now, that's not most effective. .I think there will be a lot of .argument to go to a ballot level.comparison, but that's another .term, we start to throw two .terms around, ballot level, .batch level, what does that .mean? How do we explain that in.a way that's easy for voters as .well as election officials to .understand?.I didn't answer that, but that .is .essentially taking the batch and.hand counting the batch and .comparing the totals of that to .the totals of the equipment on .that batch. . How did I do?.>> You said you're working on a .real user friendly thing, right?. >> That's the plan.. >> So you'll keep in .touch and let us know, and we .can circulate it to everybody?. >> Absolutely.. >> Awesome, thank you..Because I know I'm going to get .feedback. .Kara, could I ask you, talking .about the data that you're doing., as you know in Pennsylvania, .we're going to be transitioning .across to new systems. Can we .learn from the data that you're .collecting, where would you .start if you're advising? We're.going to have line management is.going to be changing, right, .with different systems. And .some counties are going to be .going from DRE machines to hand .marked paper ballots or some .combination. Where would you .start if you were advising a .county that was going to make a .change? What would be the first.set of steps they should take to.start the line management data .collection.. >> The first thing I .would share for anyone in .Pennsylvania is to share the .model that we use. I forgot .what R&D stood for..Robbed and /TKUP /* duplicated .as well..Their voting patterns remain the.same. They're caught off guard .with the political landscape and.changes there. But for the most.part, if you can understand .voter flow now, you can use that.data to then apply it to what .efficiencies you might be able .to get from that future .equipment. Because your voters .are still going to be the same. .They're still going to show up .at 7:00 or at 10:00 or at 7:59 .in Pennsylvania. We close at .8:00 p.m..and we always have folks rushing.in at the last minute certainly..So I would say that just you .have to start, and you can start.as soon as November, and I'd be .happy to share the tools that .we're using, and it's very easy .for the poll workers to .understand, and then also start .now getting the buy-in from our .poll workers in Pennsylvania .that we need their help with .this data..So that's the data that only .they can collect for us on .election day. . So like I had to do, I.had to rip the Band-Aid off and .jump in and start collecting the.data. It's daunting for an .election director to worry about.all the things we worry about. .Once we got the ball rolling, .it's very common for us now, and.it's a very doable program..>> Thank you for offering to .share your resources because I'm.sure it's very appreciated by .the other counties. .Can I ask one more question .before going on?.Bridgett, /* -- unless you don't.want me to ask a question..>> No, no.. >> I was curious with .the .power of story-telling, if you .have an .example of a circumstance where .the kind of process that you .were talking about with .evaluation really made a .difference from going from a .problem to a solution that had .-- you know, you talk about .short term, medium term, long .term. Do you have an example .where you can just share -- and .I'm sorry..If you don't, we'll go to .audience questions.. >> So I have a kind of.story..>> Stories are good. You can .make it up. .>> We just talked about good .data all day..So it's not so much a story .about me watching the process .work through to address a .specific solution, but I can .tell you about conversations .I've had about people -- with .people who have used it. . So one of the things .that I and my colleagues at .Auburn do in our work with the .Election Center is we spend a .lot of time talking to election .officials from around the .country about challenges they .are facing and the ways in which.they address them. And so what .I have found -- this is like a .half answer. So what I have .found, though, when through some.of the courses we present .the logic model to them, the .reaction tends to be very .positive. So on the one hand, .there's a lot of griping about, .oh, another activity that I have.to sit through and do. But then.what I see is, when people .actually sit down and think .about what it is that they .actually need to do and the .resources they're going to need .to .commit and sort of what their .actual outcome is, so just .beyond the short term, we want .people to vote, or beyond we .want people to understand what a.specific deadline is. But .thinking about the bigger .picture of how election .administration fits into this .thing that we like to call .democracy. . I've seen perhaps the .biggest impact beyond that end, .when people step outside of .themselves and their specific .circumstance and think about how.their decisions impact their .communities and states in the .country more broadly. My .non-answer..>> That was a great non-answer.. >> Thank you..>> Thank you again. Do I have .time for audience questions? I .can't actually see that far. .Okay, great. Are there audience.questions?.No audience questions. Is there.still chocolate?.>> One more thing. I just want .to let you know in Arizona we .don't have tornadoes, but we .have rain, and I just wanted you.to know, when it rains, that is .our natural disaster..I was making notes here, and I .said .I've got to make sure to ask .about tornadoes..>> Great. We do actually have .some audience questions..I'm sorry. Bright lights are a .little blinding. .>> Hi, I'm Paris. I'm a .consultant, and looking into .what all of you are doing is so .terrific..I'm a CPA from Wall Street .originally. So there's some .very interesting subjects going .on here. .One of you brought up the .question of precinct surveying, .and I was wondering have any of .you, since some of you in the .audience who have already been .on the dais have talked .about setting up your e-mail .structure with your clients and .constituency, those of you like .in Pennsylvania where you don't .have that necessarily, that .ability and stuff, how are you, .all of .you looking at communicating.? And so many online tools are .made for surveying and getting .feedback..Are we looking into that as .something that you're going to .be rolling out? Are people .rolling it out? Have they .rolled it out? Are you clear on.what I was talking about?.I'm talking about, you talked .about customer satisfaction. .You talked about getting .feedback as to what does the .voter really need?.You're kind of doing it in a .paper way, I gathered, at the .moment. But obviously an online.monkey tool or something like .that would be so efficient. Are.we looking at that kind of .communication for this industry?. >> For America, but we.actually did utilize survey .monkey. It's a unique name .there. And we got positive .feedback. We're rolling out .something new, and we really .wanted that. So we had the .ability to know who's checked .in. We go back to our voter .registration, .who has e-mails, who has texts, .or given us a cell phone, and we.responded in a short, brief .five-question, and we're looking.future forward because we can .build our psych book, and .without having to go, and, .again, not to bash third party .vendors, but we can tweak this .as much as we want without a .cost to it. We're going to .actually add. We need to slow .down our check-in process. It's.so fast that backlog..Starbucks, waiting for your .caramel macchiato ballot kind of.thing. We're going to have the .process. Take a two-question, .five-star. I know Don Palmer is.not here. We're partnering with.the bipartisan. We'll be able .to do some line studies. Have .people wear a badge, drop at the.end of the line when you come in.and help us service. . But all of that survey.stuff, we're already doing now. .I actually just got my text. .How was my early voting .experience? We've already .implemented that text -- because.I cast before I left..I should tell them how five star.the director is. I'm kidding. .Anyways, we're already .implementing. I've seen this. .I heard today on the text and .the .e-mail, so it's just more .involved. At least in Maricopa,.we are already .looking at that survey .structure..>> In Chester County, we are not.there yet frankly, but we are .jealous, and we'd like to. We .have the energy towards trying .to understand those .perspectives..In a data driven way, in .addition to maybe the anecdotal .comments that the general public.may provide. I talked a little .earlier about everyone's version.of ten minutes is very .different. So it's hard to .chase that..But if we have the data and then.maybe .the additional anecdotal after .that, that would, I think, give .us a nice marriage of .information. So not yet..>> Other questions?. >> I'm Sarah Brennan. .I'm an attorney with the voting .rights project at the ACLU. I .just had a general question. I .was really interested in .Maricopa County. When did you .guys adopt the vote centers? .And what are the parameters? .Can you go anywhere in the whole.county to use any vote center? .And I also noticed you mentioned.on .your slide you have realtime .ballot printing capacity. This .seems like such an efficiency to.prevent people going to the .wrong location. If you could .also speak to how well it .actually works. I know .sometimes realtime ballot .printing can be very time .consuming, but, of course, it .helps voters a great deal. So .I'd be interested to know a .little bit about what experience.you guys have had with that .specifically too. . >> We try not to speak.to the ACLU. Actually, we work .with Sarah. Sarah has been .working with us..But as far as the process .itself, all .of our -- the vote centers we've.been using theoretically for .almost a decade with early .voting. It's been a vote .anywhere. Early voting, we had .ballot on demand. The ballot on.demand system we had took eight .minutes to print a ballot. But .it was low turnout..So just literally in January 1 .of 2017 .when we got a new model of .ballot on demand we could do it .in 30 seconds or less. That's .what we began saying with that .kind of ability, that we can .offer this. So we added that to.an election day environment for .local. .I will say that Arizona still .has a statute that doesn't allow.for a vote anywhere on election .day for a federal election. So .in 2018 we are going to be .running a hybrid. Are there any.Arizona legislators in here? .Well, those -- no, I'm just .kidding. This is national. I'm.sorry..But the fact is it takes a lot .of data, data to prove your .process. So we have done this .through local elections. Some .of the numbers you saw were vote.anywhere local election .November, a million voters, but .still it was not a full .statewide or federal election. . This election, they .have allowed us to run a hybrid,.meaning .we'll have precinct based .consolidated locations that we .won't call vote centers. .There's two precincts showing up.to one location. That's how .we're -- not skirting the law. .That's not the right way to say .that. We don't fix elections. .But we're consolidating .locations for this 2018 to do a .hybrid, and we'll have 40 .locations that will be vote .anywhere, but we're advertising .that, if you go there, they're .not precinct based. So we're .doing a happy medium. .Right now you're correct. The .numbers you saw there why for a .vote center for all our locals, .and this '18 will be hybrid..>> Asking as a good model to .promote to other states. 30 .seconds, that's fabulous. .That's such a great efficiency, .and I'm actually fascinated. .That's great you guys are doing .a model so you will get good .data to promote perhaps a .legislative change.. >> And that's the .beauty of this data summit. .We're going to draw this data to.-- I even talked with Amber .about her great infographics she.does. We're going to rob her .graphics folks, and we need to .promote this a better way to our.legislature to say this is what .the data shows is where we want .to go.. >> Is there one more .question? Yes, looks like over .here..>> Well, what I wanted to ask, .the whole issue of evaluation, .can that .be used to also verify that a .voter voted the way they wanted .to vote, that votes can't be .changed afterwards? Like is .there a process to track that?.That's one of my main concerns, .with the evaluation process, .especially with what was spoken .of today, will that help .to verify that the election is .true and honest?.>> You want to answer that?. >> I'll answer that. .That's exactly what the audit .does. That's exactly what the .risk limiting audit does..Actually, it's looking at two .things, right? It's making sure.the voting equipment operated .the way that it was programmed .to operate..So did it tabulate the ballot .the way the ballot was marked? .The interesting thing we found .throughout the process it didn't.just validate the polling .equipment, but it also validated.our training of poll workers and.election judges in adjudicating .voter intent. That's really .critical, especially to what you.were speaking about. . If a voter marks a .ballot, and let's assume they .don't mark it exactly the way .the instructions provide..They make a mistake or they .overvote a race or some other .situation happens that it would .require a set of election judges.or poll workers to evaluate that.or make a decision..The audit verifies that was done.correctly. One thing that's .really important, I think, is .for states to think about .adopting uniform voter intent .guidelines. It's really .helpful. So that county by .county, we're not making .different decisions on what is a.correctly marked ballot..But, yes, the audit verifies .both of those.. >> If I may just add, .in front of that, similar to a .ballot status or ballot trace, .some of that lends credibility .to say, yeah, your ballot has .been received. It's in process..My next e-mail or text will be .your ballot has been verified .and counted..Now an audit then says we've .done logic and actually had it .counted. So you've got a lot of.front end for the individuals .when we start rolling these out .and people start robbing and .duplicating some of this text .process and e-mail process that .soon will be standard, I think, .with all of our election folks.. >> Great. Thank you .so much. What great questions. .And even better answers..So thank you very much to this .panel. That will conclude the .panel..I'd like to introduce closing .remarks .will be given by vice-chair .Christy McCormick. .>> So thank you all for staying .through. Those of you who have .stayed are the ones who need the.most therapy on your OCD .disorder. But let me offer my .sincere thanks to the Community .College of Philadelphia for .hosting us here today. They are.very generous with their space. .We thank them for that..Acting Secretary Robert Torres .and our partners at the .Pennsylvania Department .of State, especially Jess Meyer .and Mike Moser, who dedicated .countless hours to making this .possible and your microphone .folks as well. I want to thank .EAC staff, especially David and .Natalie and who else is here?.Brenda, Brian, so many folks on .our team helped put this .together, and I just want to .thank you for all your hard work.on that. . I want to thank all of.our moderators and panelists for.spending the time they did to .put their presentations .together. We appreciate their .time and commitment and their .expertise and contributions to .today's discussions. . And all of our .audience members, including .those of you who followed online.today. Thank you so much..Henry and your IT folks back .there too. Thank you so much. . Let me also invite you.all to check out the EAC's .website to learn more about our .recent and upcoming research and.data initiative, including .the 2018 election administration.and voting survey. There were .hard copies back there. I don't.know if there are still any left.there..You can find this information on.the EAC.gov website under the .research and data tab as well as.on our blog. .Lastly, I want to invite you all.to another event that we have in.the .works for later this month on .Tuesday, July 24th, in .Washington, D.C..I think it's at the Newseum. We.are hosting the third annual .language access for voters .summit, which is being organized.in partnership with Arizona .State University's pastor center.and the democracy fund's voice. .This will convene election .officials and language community.representatives to discuss how .election officials can meet .their language assistance .requirements under the voting .rights act and better serve .their voters with language .needs. .So with that, I just want to .thank you all for spending your .day with us..I hope you found this to be .educational and generate some .innovative ideas, and we look .forward to our next election .data summit. Thanks so much. .[APPLAUSE].

How to generate an electronic signature for the Fillabe Oregon Request To Energize An Electrical Form online

CocoSign is a browser based application and can be used on any device with an internet connection. CocoSign has provided its customers with the best method to e-sign their Fillabe Oregon Request To Energize An Electrical Form .

It offers an all in one package including validity, convenience and efficiency. Follow these instructions to put a signature to a form online:

  1. Confirm you have a good internet connection.
  2. Open the document which needs to be electronically signed.
  3. Select the option of "My Signature” and click it.
  4. You will be given alternative after clicking 'My Signature'. You can choose your uploaded signature.
  5. Design your e-signature and click 'Ok'.
  6. Press "Done".

You have successfully signed PDF online . You can access your form and email it. Excepting the e-sign alternative CocoSign proffer features, such as add field, invite to sign, combine documents, etc.

How to create an electronic signature for the Fillabe Oregon Request To Energize An Electrical Form in Chrome

Google Chrome is one of the most handy browsers around the world, due to the accessibility of a lot of tools and extensions. Understanding the dire need of users, CocoSign is available as an extension to its users. It can be downloaded through the Google Chrome Web Store.

Follow these easy instructions to design an e-signature for your form in Google Chrome:

  1. Navigate to the Web Store of Chrome and in the search CocoSign.
  2. In the search result, press the option of 'Add'.
  3. Now, sign in to your registered Google account.
  4. Access to the link of the document and click the option 'Open in e-sign'.
  5. Press the option of 'My Signature'.
  6. Design your signature and put it in the document where you pick.

After putting your e-sign, email your document or share with your team members. Also, CocoSign proffer its users the options to merge PDFs and add more than one signee.

How to create an electronic signature for the Fillabe Oregon Request To Energize An Electrical Form in Gmail?

In these days, businesses have transitted their way and evolved to being paperless. This involves the signing contract through emails. You can easily e-sign the Fillabe Oregon Request To Energize An Electrical Form without logging out of your Gmail account.

Follow the instructions below:

  1. Look for the CocoSign extension from Google Chrome Web store.
  2. Open the document that needs to be e-signed.
  3. Press the "Sign” option and design your signature.
  4. Press 'Done' and your signed document will be attached to your draft mail produced by the e-signature application of CocoSign.

The extension of CocoSign has made your life much easier. Try it today!

How to create an e-signature for the Fillabe Oregon Request To Energize An Electrical Form straight from your smartphone?

Smartphones have substantially replaced the PCs and laptops in the past 10 years. In order to made your life much easier, CocoSign give assistance to flexible your workflow via your personal mobile.

A good internet connection is all you need on your mobile and you can e-sign your Fillabe Oregon Request To Energize An Electrical Form using the tap of your finger. Follow the instructions below:

  1. Navigate to the website of CocoSign and create an account.
  2. Follow this, click and upload the document that you need to get e-signed.
  3. Press the "My signature" option.
  4. Draw and apply your signature to the document.
  5. View the document and tap 'Done'.

It takes you in an instant to put an e-signature to the Fillabe Oregon Request To Energize An Electrical Form from your mobile. Load or share your form as you wish.

How to create an e-signature for the Fillabe Oregon Request To Energize An Electrical Form on iOS?

The iOS users would be gratified to know that CocoSign proffer an iOS app to make convenience to them. If an iOS user needs to e-sign the Fillabe Oregon Request To Energize An Electrical Form , make use of the CocoSign application relivedly.

Here's advice put an electronic signature for the Fillabe Oregon Request To Energize An Electrical Form on iOS:

  1. Place the application from Apple Store.
  2. Register for an account either by your email address or via social account of Facebook or Google.
  3. Upload the document that needs to be signed.
  4. Select the section where you want to sign and press the option 'Insert Signature'.
  5. Type your signature as you prefer and place it in the document.
  6. You can email it or upload the document on the Cloud.

How to create an electronic signature for the Fillabe Oregon Request To Energize An Electrical Form on Android?

The giant popularity of Android phones users has given rise to the development of CocoSign for Android. You can place the application for your Android phone from Google Play Store.

You can put an e-signature for Fillabe Oregon Request To Energize An Electrical Form on Android following these instructions:

  1. Login to the CocoSign account through email address, Facebook or Google account.
  2. Open your PDF file that needs to be signed electronically by clicking on the "+” icon.
  3. Navigate to the section where you need to put your signature and design it in a pop up window.
  4. Finalize and adjust it by clicking the '✓' symbol.
  5. Save the changes.
  6. Load and share your document, as desired.

Get CocoSign today to make convenience to your business operation and save yourself a lot time and energy by signing your Fillabe Oregon Request To Energize An Electrical Form online.

Fillabe Oregon Request To Energize An Electrical Form FAQs

Here you can acquire solutions to the most popular questions about Fillabe Oregon Request To Energize An Electrical Form . If you have specific doubts, press 'Contact Us' at the top of the site.

Need help? Contact support

Do military personnel need money to fill out a leave request form?

No. You can go on leave and just stay in your barracks room the whole time (although you ought to not be seen during formations and whatnot or someone might decide you don’t need to be on leave anymore…)

How do I respond to a request for a restraining order? Do I need to fill out a form?

It’s a very, very serious thing a restraining order. You don’t need the right form - you NEED A LAWYER!! Get a lawyer straight away - one that knows about restraining orders…

Does a girlfriend have to fill out a leave request form for a US Army Soldier in Special Operations in Africa?

Let’s apply little common sense. Do you expect that someone without a girlfriend to work without any leave? Does that make sense for an American? Who fills out your vacation forms? A boyfriend in far off place? Answer is No. I would not expect it for an American. so, answer to original question is No. A girlfriend does NOT.. in fact, CANNOT, apply leave for a Soldier - Special Operations or not.

Easier, Quicker, Safer eSignature Solution for SMBs and Professionals

No credit card required14 days free