Recently I updated dcscrape aka simple python web scraper to allow many multiple scrapes of various data using a single request. It was limited to just basic html tags, nice links, and printed out all extracted text. Now I wanted to add more options like classes and id’s, so it has this setup but now this single function needs to become a class. So with a ‘class Dcscrape: ‘ created it can be used more universally and since I have been working a shit ton in oop style I am very comfortable refactoring it this way. Plus the single function was getting a bit messy with so many conditionals, oop the arguments will take care of this issue and be much cleaner.
For now just thinking of how I might set this up, again the beauty of this or I hope it will be able to work for many sites all in one class full of various filtering and output methods. This is also a similar style to my new arps bot class.
s1 = Dcscrape( url, tag|class|id|other )
Example:
s2 = Dcscrape( ‘https://myweathersite.com/dallas’ , ‘temps’ )
s2(‘high-temp-today’)
Result that will be returned: 105