Scraping Historical Cryptocurrency Prices [Python Beautiful Soup & Selenium Tutorial]
In this python tutorial video, we show you how you can scrape historical data for a cryptocurrency of your choice using beautifulsoup and selenium, from coinmarketcap. We capture all of the data going back as far as they have it and then automatically save it into a csv, all with python. Web scraping work with python is often times such an important early stage need for any sort of data analysis pipeline that you may have, and automated trading systems are no different.
You could use the data after scraping it, to attempt to train a predictive model for a cryptocurrency trading bot (crypto arbitrage anyone?). If you build your model properly, you may even have an improved chance at predicting future crypto prices (sounds like magic I know).
If you enjoyed the video, make sure and hit like and subscribe. We will keep putting out more great content like this on how to learn python.
Also if you have any requests for video content, just comment down below or email us (email at bottom of description).
If you don’t have python yet I would recommend installing the entire Anaconda package here –
Make sure to install the correct version for your operating system.
Once you have anaconda installed there are a few things you want to do to ensure that you are ready to follow the video tutorial. Open up the freshly installed anaconda prompt, terminal or command prompt. Enter the command ‘pip install beautifulsoup4’. Then after that completes successfully ‘pip install requests’. Finally, run `pip install selenium` then make sure to visit this link to find a chromeriver for selenium –
If after installing the above packages you still get an error, there are a few things I would recommend. First try to ‘pip install lxml’ if that is not a package you already have, this will likely quickly fix your problem. If your error persists, then in the line where you define soup (soup = bs4.BeautifulSoup(r.text, “lxml”)) just change the lxml part to be xml or html.parser. That should clear up your issue. Finally make sure that the chromedriver is in your local path or that in the line where you define driver you properly point to the location of chromedriver on your machine.
To download the example .py file in the video-
www.straightcode.net/learn/bitcoin_history_parser
Links for all documentation:
Bs4 –
Selenium –
Requests –
xPath –
[ad_2]
Source link
Hi does this still work? Or has CMC walled this data off?
bonjour, quelqu'un peut m'aider svp ?
Quand j'arrive à l'étape soup il me renvoie un tableau vide, il recupère pas les données. ça soup = bs4.BeautifulSoup(r.text, "lxml") renvoie [] (tableau vide). merci d'avance
HELLO I WOULD LIKE TO HIRE YOU TO BUILD A BOT TO TRADE STOCKS LET ME KNOW THANKS
baby just copy and paste to excel if that is just the goal …LoL… neways nice video likes!!
Prerequisite of this video?I only know basics of python.
it is only getting one month of data? not sure what you did to get the whole date set back to April 2013. can you clarify please?
thanks
hi great video. in yahoo finance for url :https://in.finance.yahoo.com/quote/AMBUJACEM.NS/history?p=AMBUJACEM.NS , its scraping only 200 rows . i checked the tr and td class names for any changes , but they are the same for rows higher up as well . think u can help ?
I am trying to scrape yahoo finance. Your method works perfectly fine but the csv file only saves data upto 101 rows. How do we increase the row size to indefinite?
Hi, doesn't work why ? Thanks in advance
with open('coinmarketExample.csv','w',encoding='utf-8',newline='')as csvfile:
TypeError: file() takes at most 3 arguments (4 given)
great tutorial, thx.
im trying to get a price from http://www.aex.com exchange but the output for 'soup.find_all('strong',{'class':'first exchange_wrap_hook'})' is "[<span class="son">–</span>]", so there is "–" instead of the numbers that should represent the price. Any ideas on how to fix it or what am I doing wrong?
Need your help.. Please give me yiur Contact number or social link
Hello there. Great video . How can you run this python selenium script daily at let's say 3 PM to do some tasks . Can we schedule it on a windows server and let is run ?