April 6-9

Monday: 12:55-2:30
Tuesday: 12:55-2:30
Wednesday: 12:25-1:47
Thursday: 12:55-2:30

     This week, I started off by relaying the past week's events to my mentor. Then, the next step in the coding process is the news. I have to decide which news I want to scrape from the high school website: the upcoming events, or the actual news stories. I am leaning towards the actual news stories because I think that those might encompass some of both categories. Next is to scrape the information I want from the website. The issue is that every news story is its own link that leads to a popup instead of a regular webpage. So, not only do I have to scrape to find each link that leads to each story, but I have to deal with the fact that each link is in an odd format due to the fact that they are all popups. Another question I have to ask myself is whether or not I want to store the articles on our database. I don't want our database too become too full, so I'd rather not have to use it for unnecessary items, however if the articles are not stored on the database, then each device will need to scrape and store the information that they need to display. Another factor is that Python, the language I am using to scrape, does not mix well with text files and HTML, the language I'd use to display the articles, does not mix well with Python. Since the it is less efficient to figure out how to locally store the articles, I decided to suck it up and store them online.
     A new problem arose after my first test: the problem with the page not loading before it is scraped has reappeared. Since I am trying to scrape from popups, the information I need is not present when I start scraping. So, I will either have to look for a new way to scrape these articles, figure out how to load the page before scraping it, or use the upcoming events for news instead. The upcoming events would not be hard to scrape, but I'm not sure that they would be as useful for users as the news articles would be.