scraping fantasy football sites... (1 Viewer)

[Follows Closely]

First Question:

I am looking for a way to get current rosters from all the major fantasy football sites out there: Yahoo, ESPN, mfl, cbs… myfantasyleague has an xml feed, but all others do not provide an api for gatering this information, so you will have to scape. Is there anything out there that I could utilize to get this information?

Second Question:

After a quick search, I have found nothing to help me with the first question. What I am thinking about developing is a service that you pass in your league url and it returns an xml document containing current rosters, etc. For some sites, such as Yahoo Private Leagues, you will also have to supply user/pass. Does this service interest anyone?

 

[Bri]

I've spent years learning scraping and spiderring and trying to develop stuff that works. Although it appears simple, it's very tricky and I wouldn't recommend going down that road.

Most of what you want to glean(gleen?) can be had by converting a cell phone WAP page into XML fairly easily. Most sites offer that.

 

[Follows Closely]

I've spent years learning scraping and spiderring and trying to develop stuff that works. Although it appears simple, it's very tricky and I wouldn't recommend going down that road.Most of what you want to glean(gleen?) can be had by converting a cell phone WAP page into XML fairly easily. Most sites offer that.

I agree that it can be tricky, especially the spider components. However, to get the roster information is very simple. In most cases one regex expression will do the trick. Actually, the biggest hurdle I foresee is all the sites that require authentication. Yahoo for example has a complicated authenication process, multiple redirects and multiple cookie checks.I usually parse/scrape the "print" view, which is usually smaller and more simple than the standard bloated HTML. Parsing the WAP is a good idea that I did not think of, WAP is usually more XML complaint, thanks.

 

[Mike]

See here: http://www.screamingsports.com

I haven't used the service but remember hearing about it during pre-season a year ago.

 

[Follows Closely]

...Actually, the biggest hurdle I foresee is all the sites that require authentication. Yahoo for example has a complicated authenication process, multiple redirects and multiple cookie checks...

I have the Java code to log into Yahoo, PM me if you are interested.