Monday, November 14, 2016

I scraped all the 2016 U.S. election data

I'm a weird guy. I enjoy building webscrapers; I find it relaxing. I have no real plans to use the 2016 U.S. election data, no particular horses to grind (I'm not to thrilled at the outcome, but hey), but I've been  hanging around /r/datasets on Reddit, and lots of people were asking for the data, and wondering if someone was going to scrape it. So, I did.

All of the data is in this Github repo. Obviously, I did not put a license of any sort on it, so feel free to use it. If you end up doing any interesting analyses with it (even boring ones, I'm not picky) I'd love to hear about it!

If I understand correctly, official verified (as opposed to reported) results will start to be released by the states in 2-4 weeks, but it seemed a shame the data was not, as far as I could tell, easily available right now.

(Of course, given my experience as to how the universe works, probably there will be a better data dump of all this info somewhere soon, or someone will point out it already exists somewhere my Google Fu wasn't strong enough to find, making my 10-12 hours of effort redundant. But it wouldn't have happened if I hadn't forced fate's hand!)

Is scraping legal? Yes. Is it ethical? Pretty much. Feel free to disagree; my position is that this is all public data, not in any copyright, presented publicly, and I scraped it by automating an actual web browser, so I did not use up any more of the websites' resources than a regular visitor. If I have breached any terms of service, I'll just have to live with the consequences, of which there are likely to be none. Here's a good Quora post about the subject.





• • •

107 comments:

  1. Hi David, you may be weird for enjoying web scraping, but you've made at least one geek very, very happy...
    I'll use your data for some coursework I'm doing for my MSc in Data Science if that's OK? I'd just started trying to get a dataset for the US elections together when I stumbled on your rather tasty dataset. Aiming to discover which factors influenced voting in the election. I'll be assembling some extra data on demographics, prior voting patterns, economic performance and a few others and then doing some machine learning and visualisations in search of some answers. Should you ever be in London (UK, not ON), I think I owe you a few beers... best, Peter

    ReplyDelete
    Replies
    1. I'm overjoyed you can make use of my efforts! Do so with my blessing!

      Delete
    2. Thanks David! I will share what conclusions I reach with you when done... might take a few weeks though. All the best, Peter

      Delete
  2. Hi David, coursework reports done, thanks again for that data! If you're suffering from insomnia and want a look, let me know the best way of getting them over to you -- I don't really want to post them openly...

    ReplyDelete
  3. This post is much helpful for us. This is really very massive value to all the readers and it will be the only reason for the post to get popular with great authority.
    Web Designing Training in Chennai

    ReplyDelete
  4. great activities are conducting to students. this makes some relieving to them technical studies. i wish to follow and conduct those kind of activities you are mentioning. keep share.
    PTE Training in Chennai

    ReplyDelete
  5. Thanks for this blog. provided great information. All the details are explained clearly with the great explanation. Thanks for this wonderful blog. Step by step processes execution are given clearly.Know the details about different thing.
    Digital Marketing Company in Chennai

    ReplyDelete
  6. Great post! I am actually getting ready to across this information, It's very helpful for this blog.Also great with all of the valuable information you have Keep up the good work you are doing well.
    Security Services in Chennai

    ReplyDelete
  7. nice blog too informative. looking and reading your points its so impressive. doing more blog like this. i really appreciated doing like this.
    SEO Company in India

    ReplyDelete
  8. UFC 209: Woodley vs. Thompson 2 is an upcoming mixed martial arts event produced by the Ultimate Fighting Championship that will be held on March 4, 2017. Event information, results, video, and fighter information for UFC 209

    UFC 209


    UFC 209 Fight Card


    UFC 209 Live


    UFC 209 Live Stream


    UFC 209 Start Time


    UFC 209


    UFC 209 Live

    ReplyDelete
  9. Thank you for taking the time to provide us with your valuable information. We strive to provide our candidates with excellent care and we take your comments to heart.As always, we appreciate your confidence and trust in us
    Digital Marketing Company in India

    ReplyDelete
  10. We give a moment of silence for these five matches that we anticipated but did not make it to the final WWE WrestleMania 33 lineup. At WWE Wrestlemania 33, Taker will step into the ring with Roman Reigns, who has promised repeatedly that he will retire The Deadman. WWE WrestleMania 33: 10 Returns That Could Happen ... As such, Michaels was given his first Wrestlemania match.

    Wrestlemania

    WWE Wrestlemania

    Wrestlemania 33

    WWE Wrestlemania 33

    Wrestlemania 33

    Wrestlemania 2017

    WWE Wrestlemania 2017

    Wrestlemania Live

    WWE Wrestlemania Live

    Wrestlemania Live Stream

    WWE Wrestlemania Live Stream

    Wrestlemania 33 Live

    Wrestlemania 33 Live Stream

    WWE Wrestlemania 33 Live

    WWE Wrestlemania 33 Live Stream


    Get WWE WrestleMania 33 results & updates, including 2017 winners and WWE video highlights of the best matches from WrestleMania 33 this year.

    ReplyDelete
  11. The Masters Tournament, also known as The Masters or The US Masters, is one of the four major championships in professional golf. The Masters is scheduled for the first full week of April, and it is the first of the majors to be played each year.

    Master Golf 2017

    2017 Master Golf

    Masters Tournament

    Master Tournament

    Masters Tournament 2017

    2017 Masters Golf Tournament

    Masters Golf Tournament 2017

    Masters Golf Tournament

    Master Golf 2017

    2017 Master Golf

    Masters Tournament

    Master Tournament

    Masters Tournament 2017

    2017 Masters Golf Tournament

    Masters Golf Tournament 2017

    Masters Golf Tournament

    Masters Tournament 2017 live coverage from Augusta National Golf Club at CBSSports.com. Watch the tournament live, choose cameras and get live stats.

    ReplyDelete
  12. Thanks To Watch The Masters 2017 Live Stream, one of the most prestigious tournaments in golf, is set to take place this week at Augusta National Golf Club from April 6-9.

    Masters 2017

    The Masters 2017

    Masters 2017

    The Masters 2017

    Masters Golf

    UFC 210

    UFC 210

    We already know Tiger Woods won’t be Masters 2017 Live participating in the 2017 Masters, but there are still intriguing questions entering this year’s tournament.

    ReplyDelete
  13. A week after Chavez vs Canelo went toe-to-toe in an instant classic, boxing fans get another star-studded fight when Saul “Canelo” Alvarez (48-1-1, 34 KOs) and Julio Cesar Chavez Jr. (50-2-1, 32 KOs) meet at the T-Mobile Arena in Las Vegas.
    Canelo vs Chavez Jr

    Chavez Jr vs Canelo

    Chavez Jr vs Canelo

    ReplyDelete

  14. you are posting a good information for people and keep maintain and give more update too.
    seo company in india

    ReplyDelete
  15. The SummerSlam 2017 is an upcoming professional wrestling pay-per-view (PPV) event and WWE Network event produced by WWE for the Raw and WWE SummerSlam brands. It will take place on August 20, 2017, at the Barclays Center in Brooklyn, New York. It will be the thirtieth event under the SummerSlam chronology.
    WWE SummerSlam 2017

    SummerSlam Live

    WWE SummerSlam Live

    Watch SummerSlam

    Watch WWE SummerSlam

    SummerSlam live Stream

    WWE SummerSlam Live Stream

    ReplyDelete
  16. It is really very amazing for me that you find this interesting practicing your written english and entertaining and you are happy to do it.

    ReplyDelete
  17. Very nice post.really I apperciate your blog.Thanks for sharing.keep sharing more blogs.

    ดูหนังออนไลน์

    ReplyDelete
  18. Omrdeals.in is an online platform that allows people to find offers and the greatest deals in and around OMR. It is the directory of the business in and around omr. It helps internet users to surf the required deals and offers.

    offers in Chennai

    ReplyDelete
  19. It is a great job that you are working and offer our necessary need to your posts. We like your job and wish you all the best.
    digital marketing company in india

    ReplyDelete
  20. Benefits of reading your post. This will help in the maintenance of our health.The daily routine will help you lose weight quickly and safely.My life is totally transformed after I followed this diet.I feeling great about myself.

    Herbalife in Chennai
    Nutritionclub in Chennai
    Weight Loss in Chennai
    Weight Gain in Chennai

    ReplyDelete
  21. In the last few months we've seen a lot of Health Care Reform rules and regulations being introduced by the Health and Human Services Department. Every time that happens, the media gets hold of it and all kinds of articles are written in the Wall Street Journal, the New York Times, and the TV network news programs talk about it. All the analysts start talking about the pros and cons, and what it means to businesses and individuals. Health is God

    ReplyDelete
  22. I think this is a real great article post.Really looking forward to read more. Visit at
    Crazy Video Hub

    ReplyDelete
  23. Nutra Trials defines personal characteristics of different health products including skincare, weight loss, muscle and male enhancement. The study presented here is briefly described for reader convenience and to deliver them assurance with health standards. The best possible answers are given here regarding the selection of an ideal supplement or cream or serum that possibly remains to be safe for health and do not cause any side effects.

    ReplyDelete
  24. Wow it is really wonderful and awesome thus it is very much useful for me to understand many concepts and helped me a lot. it is really explainable very well and i got more information from your blog.

    Data Science Training in Chennai
    Data science training in bangalore
    Data science online training
    Data science training in pune
    Data Science training in kalyan nagar
    Data Science training in OMR
    selenium training in chennai

    ReplyDelete
  25. Great post! I am actually getting ready to across this information, It’s very helpful for this blog.Also great with all of the valuable information you have Keep up the good work you are doing well.

    Data Science with Python training in chenni
    Data Science training in chennai
    Data science training in velachery
    Data science training in tambaram
    Data Science training in anna nagar
    Data Science training in chennai
    Data science training in Bangalore

    ReplyDelete
  26. I really like the dear information you offer in your articles. I’m able to bookmark your site and show the kids check out up here generally. Im fairly positive theyre likely to be informed a great deal of new stuff here than anyone
    java training in tambaram | java training in velachery

    java training in omr | oracle training in chennai

    java training in annanagar | java training in chennai

    ReplyDelete
  27. This is a great post ! it was very informative. I look forward in reading more of your work. Also, I made sure to bookmark your website so I can come back later. I enjoyed every moment of reading it..
    kim kardashian sex tape
    porn sex video hd
    mia khalifa sex video
    sunny leone sexy movie

    ReplyDelete
  28. I would assume that we use more than the eyes to gauge a person's feelings. Mouth. Body language. Even voice. You could at least have given us a face in this test.
    python training in pune
    python online training
    python training in OMR

    ReplyDelete
  29. I would really like to read some personal experiences like the way, you've explained through the above article. I'm glad for your achievements and would probably like to see much more in the near future. Thanks for share.
    python training in pune
    python online training
    python training in OMR

    ReplyDelete
  30. This is most informative and also this post most user friendly and super navigation to all posts... Thank you so much for giving this information to me.. 
    Devops training in velachery
    Devops training in annanagar
    Devops training in sholinganallur

    ReplyDelete
  31. The knowledge of technology you have been sharing thorough this post is very much helpful to develop new idea. here by i also want to share this.
    Devops training in tambaram
    DevOps online Training
    DevOps Training in USA

    ReplyDelete
  32. Your good knowledge and kindness in playing with all the pieces were very useful. I don’t know what I would have done if I had not encountered such a step like this.
    Blueprism training institute in Chennai

    Blueprism online training

    Blue Prism Training Course in Pune

    Blue Prism Training Institute in Bangalore

    ReplyDelete
  33. As you have now understood the usage of ‘Record and Playback’ tool, the following are the different posts using which you can explore the functioning of ‘Selenium IDE’
    selenium Training in chennai

    ReplyDelete
  34. All the latest updates from the Python Automationminds team. Python Automationminds lets you program in Python, in your browser. No need to install any software, just start coding straight away. There's a fully-functional web-based console and a programmer's text-editor
    Phyton training in Chennai

    ReplyDelete
  35. Good Post! Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge as updated one, keep blogging.

    angularjs Training in chennai
    angularjs Training in chennai

    angularjs-Training in tambaram

    angularjs-Training in sholinganallur

    angularjs-Training in velachery

    ReplyDelete
  36. Gaining Python certifications will validate your skills and advance your career.
    python certification

    ReplyDelete
  37. I found your blog while searching for the updates, I am happy to be here. Very useful content and also easily understandable providing.. Believe me I did wrote an post about tutorials for beginners with reference of your blog. 
    angularjs Training in chennai
    angularjs-Training in pune

    angularjs-Training in chennai

    angularjs Training in chennai

    angularjs-Training in tambaram

    ReplyDelete
  38. Greetings. I know this is somewhat off-topic, but I was wondering if you knew where I could get a captcha plugin for my comment form? I’m using the same blog platform like yours, and I’m having difficulty finding one? Thanks a lot.
    AWS Interview Questions And Answers
    AWS Online Training | Online AWS Certification Course - Gangboard
    AWS Training in Chennai | AWS Training Institute in Chennai Velachery, Tambaram, OMR
    AWS Training in Bangalore |Best AWS Training Institute in BTM ,Marathahalli

    ReplyDelete
  39. Greetings. I know this is somewhat off-topic, but I was wondering if you knew where I could get a captcha plugin for my comment form? I’m using the same blog platform like yours, and I’m having difficulty finding one? Thanks a lot.
    AWS Interview Questions And Answers
    AWS Online Training | Online AWS Certification Course - Gangboard
    AWS Training in Chennai | AWS Training Institute in Chennai Velachery, Tambaram, OMR
    AWS Training in Bangalore |Best AWS Training Institute in BTM ,Marathahalli

    ReplyDelete
  40. Greetings. I know this is somewhat off-topic, but I was wondering if you knew where I could get a captcha plugin for my comment form? I’m using the same blog platform like yours, and I’m having difficulty finding one? Thanks a lot.
    AWS Interview Questions And Answers
    AWS Online Training | Online AWS Certification Course - Gangboard
    AWS Training in Chennai | AWS Training Institute in Chennai Velachery, Tambaram, OMR
    AWS Training in Bangalore |Best AWS Training Institute in BTM ,Marathahalli

    ReplyDelete
  41. how to watch World Series 2018 live The 2018 Major League Baseball is set to begin on October 23, and a potential Game 7 is scheduled for October 31.
    https://worldserieshints.com/

    ReplyDelete

  42. The blog you had post is verymuch useful for us to know about the Web designing. thanks for your information sharing with us.
    Web Designing Institute in Coimbatore
    Web Designing Course
    Web Design Classes
    Website Design Course
    Learning Web Design

    ReplyDelete
  43. I am happy to find this post Very useful for me, as it contains lot of information

    Article submission sites
    Education

    ReplyDelete
  44. The 2018 Country Music Association has announced that the 52nd annual CMA Awards Live Stream. Country Music Association it's called CMA. annual music awards program on network broadcast here.

    ReplyDelete
  45. Amazing information,thank you for your ideas.after along time i have studied
    an interesting information's.we need more updates in your blog.
    Java Training in Karapakkam
    Java Training Institutes in Numgambakkam
    Java Courses in Chennai Perambur
    Best Java Training in Bangalore

    ReplyDelete
  46. Hello, I am thomus jons thank you for this informative post. That is a great job. Wish you more success.Thank you so much and for you all the best. Takes Down
    123movies

    ReplyDelete
  47. Very useful and informative content has been shared out here, Thanks for sharing it.
    Visit Learn Digital Academy for more information on Digital marketing course in Bangalore https://www.learndigital.co/.

    ReplyDelete
  48. Times For Health is Online Health & Wellness Program! This post is really really nice and so informative. Thanks for the nice post share with us! https://www.timesforhealth.com/

    ReplyDelete
  49. TecSmash is your ultimate source of Technology news and Make Money Online product reviews. We research and review all Tech, MMO, Biz Opp and IM products.TecSmash

    ReplyDelete
  50. Best softwares for Internet Marketers and legitimate make money online opportunities cxyrc

    ReplyDelete
  51. Income designers.com is the number one destination to find genuine make money online programs and services. Income designers

    ReplyDelete

  52. Healrun is a health news blog we provide the latest news about health, Drugs and latest Diseases and conditions. We update our users with health tips and health products reviews. If you want to know any information about health or health product (Side Effects & Benefits) Feel Free To ask HealRun Support Team.

    ReplyDelete
  53. We are here to give you a complete review on the Parallel Profit project by Steve Clayton and Aidan Booth. If you are someone from the field you would already be familiar with these two names, for those of who are new. Parallel Profits Review

    ReplyDelete

  54. Pilpedia is supplying 100 percent original and accurate information at each moment of time around our site and merchandise, and the intent is to improve the usage of good and pure health supplement. For More Info please visit Pilpedia

    ReplyDelete
  55. Supplements For Fitness supplements because they are not informed about them. A pharmacist can also tell you if a particular supplement will interfere with any medication you are taking.

    https://www.supplementsforfitness.com/

    ReplyDelete
  56. Found your post interesting to read. I cant wait to see your post soon. Good Luck for the upcoming update.This article is really very interesting and effective.
    Blueprism Training in pune
    blue prism training

    ReplyDelete
  57. Whoa! I’m enjoying the template/theme of this website. It’s simple, yet effective. A lot of times it’s very hard to get that “perfect balance” between superb usability and visual appeal. I must say you’ve done a very good job with this.

    AWS TRAINING IN BANGALORE|NO.1AWS TRAINING INSTITUTES IN BANGALORE

    ReplyDelete
  58. This comment has been removed by the author.

    ReplyDelete