Website data

Discussion in 'Ask the Admin' started by shmmeee, Oct 24, 2017.

  1. shmmeee

    shmmeee Well-Known Member

    Joined:
    Jul 11, 2011
    Messages:
    13,323
    Likes Received:
    9,372
    Location:
    Coventry, United Kingdom, United Kingdom
    Nick:

    I was thinking about playing around with some data from the site, like post sentiment or like graphs. Just to see if I can and because I thought the results would be fun (who likes who, who is the happiest and grumpiest poster).

    Do you have access to that, or if not would there be an issue with me writing a scraper to get it, or would that hurt your hosting costs/run up against anti-bot measures?

    No promises on output, I might get bored and wander off halfway through. But if you’re OK with it I’ll have a crack.
     
    Otis likes this.
  2. Nick

    Nick Administrator

    Joined:
    Feb 25, 2008
    Messages:
    74,428
    Likes Received:
    23,665
    Location:
    Coventry
    I can try :)

    What sort of data do you need? Send me a PM and happy to try and find things.

    Depending on the scraper it might get blocked by security if it looks dodgy and hammers the site :)
     
  3. shmmeee

    shmmeee Well-Known Member

    Joined:
    Jul 11, 2011
    Messages:
    13,323
    Likes Received:
    9,372
    Location:
    Coventry, United Kingdom, United Kingdom
    Bump.

    Thought I’d have a go at working out a league table of likes per post just for fun. I was gonna just scrape the data but the members page only seems to show the top twenty or so for likes and posts. Is there a full member list somewhere that has how many posts and how many likes each user has?
     
  4. Nick

    Nick Administrator

    Joined:
    Feb 25, 2008
    Messages:
    74,428
    Likes Received:
    23,665
    Location:
    Coventry
    What format do you want? If it's just username, number of posts and number of likes I could probably sort something.
     
  5. shmmeee

    shmmeee Well-Known Member

    Joined:
    Jul 11, 2011
    Messages:
    13,323
    Likes Received:
    9,372
    Location:
    Coventry, United Kingdom, United Kingdom
    Yeah a csv would be perfect.

    As that was so easy, is there any way to view back past 2016 for who gave you a like? Then I could do who likes who as well.
     
  6. Nick

    Nick Administrator

    Joined:
    Feb 25, 2008
    Messages:
    74,428
    Likes Received:
    23,665
    Location:
    Coventry
    Dont think that would be as simple as Id need to query every post to see who it was who liked it and then tally it all up.

    Who works with CSV any more? It's all JSON nowadays ;)
     
  7. shmmeee

    shmmeee Well-Known Member

    Joined:
    Jul 11, 2011
    Messages:
    13,323
    Likes Received:
    9,372
    Location:
    Coventry, United Kingdom, United Kingdom
    You can give it as json if you like. As long as it’s not XML, I’m working with that at work as we speak and I am not a fan.
     
  8. Nick

    Nick Administrator

    Joined:
    Feb 25, 2008
    Messages:
    74,428
    Likes Received:
    23,665
    Location:
    Coventry
    Will have a look when I get chance, need an hour or so to get the query right.
     
  9. jimmyhillsfanclub

    jimmyhillsfanclub Well-Known Member

    Joined:
    May 31, 2012
    Messages:
    3,516
    Likes Received:
    3,249
    Location:
    here
    I'll have a quick guess on the likes per post before shmmeee publishes results:

    1: oldskyblue58
    2. 87mint
    3. hill83

    if im right, i'm going to add a 2 digit number to my username as its clearly making the difference....

    .......and absolutely nothing to do with cutting financial analysis, witty match reports or hilarious giffery...
     
    Captain Dart, Otis and duffer like this.
  10. B-Ban-Boogie

    B-Ban-Boogie Active Member

    Joined:
    Dec 6, 2011
    Messages:
    253
    Likes Received:
    107
    Location:
    Bradenton, FL
    I'm guessing sw88 is up there too!!
     
    Otis, jimmyhillsfanclub and duffer like this.

Share This Page