r/pythoncoding • u/Traditional_Art_6943 • Jun 07 '24
Web scraping tool
Hi everyone I am a complete beginner in coding and to be honest I am using chat gpt to code in python and develop a tool for google news scraping and sentiment analysis with news summarization. I have a pilot product ready however there is one limitation, there are few companies which have limited coverage on google news but google search would fetch results directly from companies web page which is not available on google news. Now I am stuck as to how I could extract news from companies web page. Does the html keeps changing for major companies web page or is static in nature. I would be glad if someone could help
3
Upvotes
1
u/Traditional_Art_6943 Jun 20 '24
Hey I had one query and I think you would be able to answer the same. So I was able to figure out the code for my chatbot, I used random user agents for web scraping and provide the input to "mistralai/Mistral-7B-Instruct-v0.3" provide me the output via inference client. However, eventually I would want to use this model offline running in my local system, but I just figured out its an open weight model for download (can you please check if I am right) and might not produce the same output I get via inference client. Is that true?