Advanced search

Message boards : News : CSG now exporting stats

Author Message
Travis Desell
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 16 Jan 12
Posts: 1813
Combined Credit: 23,514,257
DNA@Home: 293,563
SubsetSum@Home: 349,212
Wildlife@Home: 22,871,482
Wildlife@Home Watched: 212,926s
Wildlife@Home Events: 51
Climate Tweets: 21
Images Observed: 774

              
Message 4545 - Posted: 27 Sep 2014, 19:02:50 UTC
Last modified: 27 Sep 2014, 19:06:07 UTC

Looks like I have the updated stats export working.

the user, team and host files now contain additional XML for the per-project credit.

for example:

< name>Travis Desell< / name> < country>United States< / country> < create_time>1326744021< / create_time> < total_credit>1008951.056735< / total_credit> < expavg_credit>103.127181< / expavg_credit> < expavg_time>1411308097.997700< / expavg_time> < dna_total_credit>279427.000000< / dna_total_credit> < dna_expavg_credit>103.000000< / dna_expavg_credit> < dna_expavg_time>1411308097.000000< / dna_expavg_time> < sss_total_credit>343868.000000< / sss_total_credit> < sss_expavg_credit>0.000000< / sss_expavg_credit> < sss_expavg_time>1372797001.000000< / ss_expavg_time> < wildlife_total_credit>385655.000000< / wildlife_total_credit> < wildlife_expavg_credit>25.000000< / wildlife_expavg_credit> < wildlife_expavg_time>1407964455.000000< / wildlife_expavg_time>

(note the spaces in the XML are only so they show up in the forum post).

The total_credit, expavg_credit, expavg_time are the combined stats for all the sub projects. Each subproject has it's own set of xml for total_credit, expavg_credit and expavg_time.

The xml is the same for users, hosts and teams. Hope this works -- let me know if there are any issues importing this, or if any changes need to be made.

thanks!
--Travis

Bok
Send message
Joined: 31 Jan 12
Posts: 16
Combined Credit: 3,482,574
DNA@Home: 170,243
SubsetSum@Home: 327,733
Wildlife@Home: 2,984,598
Wildlife@Home Watched: 158s
Wildlife@Home Events: 0
Climate Tweets: 0
Images Observed: 0

      
Message 4547 - Posted: 27 Sep 2014, 19:54:50 UTC - in response to Message 4545.

hmmm,

Not really sure why you have done it this way. It would mean each stats site would have to write a custom parser just for this project. Quite a bit of work.

Why didn't you just use the per-app credit system that's been recently added? (BitcoinUtopia are using it successfully). Then leave csg as the main export in standard xml output and have the per-app credit in the user_work and team_work files done by the main server code.

Just my 2 cents.

Travis Desell
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 16 Jan 12
Posts: 1813
Combined Credit: 23,514,257
DNA@Home: 293,563
SubsetSum@Home: 349,212
Wildlife@Home: 22,871,482
Wildlife@Home Watched: 212,926s
Wildlife@Home Events: 51
Climate Tweets: 21
Images Observed: 774

              
Message 4549 - Posted: 27 Sep 2014, 20:17:52 UTC - in response to Message 4547.

hmmm,

Not really sure why you have done it this way. It would mean each stats site would have to write a custom parser just for this project. Quite a bit of work.

Why didn't you just use the per-app credit system that's been recently added? (BitcoinUtopia are using it successfully). Then leave csg as the main export in standard xml output and have the per-app credit in the user_work and team_work files done by the main server code.

Just my 2 cents.


Didn't know it made it into the main branch yet. Talked to DA and he said it wasn't there about a week ago. I can try and update things to use that instead.

Bok
Send message
Joined: 31 Jan 12
Posts: 16
Combined Credit: 3,482,574
DNA@Home: 170,243
SubsetSum@Home: 327,733
Wildlife@Home: 2,984,598
Wildlife@Home Watched: 158s
Wildlife@Home Events: 0
Climate Tweets: 0
Images Observed: 0

      
Message 4595 - Posted: 29 Sep 2014, 16:11:42 UTC

I'm conflicted on what to do with these stats over at free-dc. So, interested to get opinions.

I can

a) add the csg stats, also parsing the subproject data, but leave the deprecated projects out there also (which would mean duplicate credits for those that crunched prior to csg)
b) add the csg stats and remove the data from the deprecated projects.
c) not add csg (not really an option as I try to include every BOINC project regardless)

Reasons for a) are that I also track what is called MM's which are metadata quantifiers for things like how many 100k's, 1M's users have across projects. If I remove the deprecated projects, users will lose some of these.

One other option I could consider is to do b) but then include subproject data in the MM calculation, meaning CSG could over up to 4 targets. (one overall and on each for the 3 subprojects).

Bok

Travis Desell
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 16 Jan 12
Posts: 1813
Combined Credit: 23,514,257
DNA@Home: 293,563
SubsetSum@Home: 349,212
Wildlife@Home: 22,871,482
Wildlife@Home Watched: 212,926s
Wildlife@Home Events: 51
Climate Tweets: 21
Images Observed: 774

              
Message 4596 - Posted: 29 Sep 2014, 16:31:02 UTC - in response to Message 4595.

I'm conflicted on what to do with these stats over at free-dc. So, interested to get opinions.

I can

a) add the csg stats, also parsing the subproject data, but leave the deprecated projects out there also (which would mean duplicate credits for those that crunched prior to csg)
b) add the csg stats and remove the data from the deprecated projects.
c) not add csg (not really an option as I try to include every BOINC project regardless)

Reasons for a) are that I also track what is called MM's which are metadata quantifiers for things like how many 100k's, 1M's users have across projects. If I remove the deprecated projects, users will lose some of these.

One other option I could consider is to do b) but then include subproject data in the MM calculation, meaning CSG could over up to 4 targets. (one overall and on each for the 3 subprojects).

Bok


I'm fine with whatever keeps the most people happy. What are you doing with other sites with sub-projects? With the per-app code in BOINC now it might become more common...

Bok
Send message
Joined: 31 Jan 12
Posts: 16
Combined Credit: 3,482,574
DNA@Home: 170,243
SubsetSum@Home: 327,733
Wildlife@Home: 2,984,598
Wildlife@Home Watched: 158s
Wildlife@Home Events: 0
Climate Tweets: 0
Images Observed: 0

      
Message 4599 - Posted: 29 Sep 2014, 17:06:25 UTC - in response to Message 4596.
Last modified: 29 Sep 2014, 17:09:12 UTC

I display it and track it in various places.

http://stats.free-dc.org/stats.php?page=user&proj=bit&name=25

Scroll down a little to see subproject breakdown, can then click on the subproject to see top lists etc.

Some other projects have been doing it for a while with their own code. PrimeGrid, Yoyo@Home and NFS@Home. Makes it easier when it's standardized though. ( I would personally like to see the user_work and team_work described in the db_dump.xml)

Werinbert
Send message
Joined: 24 Aug 14
Posts: 17
Combined Credit: 1,000,279
DNA@Home: 300,319
SubsetSum@Home: 699,960
Wildlife@Home: 0
Wildlife@Home Watched: 25,839s
Wildlife@Home Events: 20
Climate Tweets: 5
Images Observed: 0

        
Message 4607 - Posted: 29 Sep 2014, 23:29:51 UTC

I have been putting off merging my SubsetSum accounts until the dust has settled. I will admit that I am a glutton for projects so I am leaning towards option (a), but willing to go anyway.

Profile Conan
Avatar
Send message
Joined: 13 Apr 12
Posts: 151
Combined Credit: 47,672,899
DNA@Home: 399,792
SubsetSum@Home: 1,448,876
Wildlife@Home: 45,824,231
Wildlife@Home Watched: 70,910s
Wildlife@Home Events: 0
Climate Tweets: 412
Images Observed: 0

          
Message 4618 - Posted: 30 Sep 2014, 7:31:41 UTC - in response to Message 4595.

I'm conflicted on what to do with these stats over at free-dc. So, interested to get opinions.

I can

a) add the csg stats, also parsing the subproject data, but leave the deprecated projects out there also (which would mean duplicate credits for those that crunched prior to csg)
b) add the csg stats and remove the data from the deprecated projects.
c) not add csg (not really an option as I try to include every BOINC project regardless)

Reasons for a) are that I also track what is called MM's which are metadata quantifiers for things like how many 100k's, 1M's users have across projects. If I remove the deprecated projects, users will lose some of these.

One other option I could consider is to do b) but then include subproject data in the MM calculation, meaning CSG could over up to 4 targets. (one overall and on each for the 3 subprojects).

Bok


How about a modified version of a)
Add CSG and parse the (now) sub-projects data as you suggest but add say an * after the CSG listing to indicate that the extra data for MMs is now listed under the CSG project?
That way you wont have the duplicate credit problem you allude to.

Would that be workable?

Just a thought.

Conan

Bok
Send message
Joined: 31 Jan 12
Posts: 16
Combined Credit: 3,482,574
DNA@Home: 170,243
SubsetSum@Home: 327,733
Wildlife@Home: 2,984,598
Wildlife@Home Watched: 158s
Wildlife@Home Events: 0
Climate Tweets: 0
Images Observed: 0

      
Message 4619 - Posted: 30 Sep 2014, 11:14:35 UTC - in response to Message 4618.



How about a modified version of a)
Add CSG and parse the (now) sub-projects data as you suggest but add say an * after the CSG listing to indicate that the extra data for MMs is now listed under the CSG project?
That way you wont have the duplicate credit problem you allude to.

Would that be workable?

Just a thought.

Conan


I don't really understand what you are proposing here??

Profile Conan
Avatar
Send message
Joined: 13 Apr 12
Posts: 151
Combined Credit: 47,672,899
DNA@Home: 399,792
SubsetSum@Home: 1,448,876
Wildlife@Home: 45,824,231
Wildlife@Home Watched: 70,910s
Wildlife@Home Events: 0
Climate Tweets: 412
Images Observed: 0

          
Message 4621 - Posted: 30 Sep 2014, 21:45:40 UTC

For my proposal I am suggesting that the 3 Citizen Science Grid sub-projects (SubsetSum, DNA, Wildlife) be shown under the Citizen Science Grid listing, as you do for Primegrid, yoyo and others, and removed from the main listing as individual projects to stop the duplication of credits.

I am also proposing that some sort of indicator be placed next to the Citizen Science Grid listing on the main page with a reference to where those 3 projects now are and the fact they are still counted towards your MM data.

That indicator could be anything, the usual one is a * .

So that Citizen Science Grid * is the listing and the * is also placed at the top or bottom of the listing with its information about Citizen Science Grid and its sub-projects of SubsetSum, DNA , Wildlife.

I hope that makes a bit more sense, it is hard to put into words, well at least for me anyway.

Conan

Travis Desell
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 16 Jan 12
Posts: 1813
Combined Credit: 23,514,257
DNA@Home: 293,563
SubsetSum@Home: 349,212
Wildlife@Home: 22,871,482
Wildlife@Home Watched: 212,926s
Wildlife@Home Events: 51
Climate Tweets: 21
Images Observed: 774

              
Message 4623 - Posted: 1 Oct 2014, 0:16:25 UTC - in response to Message 4621.

Would there be any way to just have Citizen Science Grid listed, but the names of the sub projects nearby?

Something like:

Citizen Science Grid
---- DNA@Home
---- SubsetSum@Home
---- Wildlife@Home

Or

Citizen Science Grid (DNA@Home, SubsetSum@Home, Widlife@Home)

?

Bok
Send message
Joined: 31 Jan 12
Posts: 16
Combined Credit: 3,482,574
DNA@Home: 170,243
SubsetSum@Home: 327,733
Wildlife@Home: 2,984,598
Wildlife@Home Watched: 158s
Wildlife@Home Events: 0
Climate Tweets: 0
Images Observed: 0

      
Message 4627 - Posted: 1 Oct 2014, 2:00:49 UTC

I don't think either of those *fix* the issue unfortunately.

But I think what I'm going to do (though still thinking about it) is to have my main credit MM's include subprojects also. This will mean I'll deprecate the wildlife,subsetsum and dna stats (they are in the subproject data for csg anyway) and there can be a lot more entries eligible for the MM's as it will also include the various Primegrid subprojects, yoyo@home projects etc.

I do need a few days to fully think about it and it needs a table change to include cpid on my subproject table (which unfortunately is not on the user_work table, but that's a minor issue as I can link it in).

For the meantime, i'll leave the other projects in as well so there is some duplicate credit, but this will eventually go away..

lionelc
Send message
Joined: 20 Dec 12
Posts: 9
Combined Credit: 150,433
DNA@Home: 100,173
SubsetSum@Home: 50,163
Wildlife@Home: 97
Wildlife@Home Watched: 1,654s
Wildlife@Home Events: 0
Climate Tweets: 0
Images Observed: 0

    
Message 4628 - Posted: 1 Oct 2014, 4:59:43 UTC - in response to Message 4627.

Could we simply have the 3 projects as it was before?:

DNA@Home
SubsetSum@Home
Wildlife@home

Nothing more, nothing less

lionelc
Send message
Joined: 20 Dec 12
Posts: 9
Combined Credit: 150,433
DNA@Home: 100,173
SubsetSum@Home: 50,163
Wildlife@Home: 97
Wildlife@Home Watched: 1,654s
Wildlife@Home Events: 0
Climate Tweets: 0
Images Observed: 0

    
Message 4629 - Posted: 1 Oct 2014, 5:00:03 UTC - in response to Message 4627.

Could we simply have the 3 projects as it was before?:

DNA@Home
SubsetSum@Home
Wildlife@home

Nothing more, nothing less

lionelc
Send message
Joined: 20 Dec 12
Posts: 9
Combined Credit: 150,433
DNA@Home: 100,173
SubsetSum@Home: 50,163
Wildlife@Home: 97
Wildlife@Home Watched: 1,654s
Wildlife@Home Events: 0
Climate Tweets: 0
Images Observed: 0

    
Message 4630 - Posted: 1 Oct 2014, 5:00:55 UTC - in response to Message 4627.

Could we simply have the 3 projects as it was before?:

DNA@Home
SubsetSum@Home
Wildlife@home

Nothing more, nothing less

lionelc
Send message
Joined: 20 Dec 12
Posts: 9
Combined Credit: 150,433
DNA@Home: 100,173
SubsetSum@Home: 50,163
Wildlife@Home: 97
Wildlife@Home Watched: 1,654s
Wildlife@Home Events: 0
Climate Tweets: 0
Images Observed: 0

    
Message 4631 - Posted: 1 Oct 2014, 5:00:59 UTC - in response to Message 4627.

Could we simply have the 3 projects as it was before?:

DNA@Home
SubsetSum@Home
Wildlife@home

Nothing more, nothing less

Dirk Broer
Send message
Joined: 7 Aug 14
Posts: 65
Combined Credit: 8,209,593
DNA@Home: 285,955
SubsetSum@Home: 1,566,395
Wildlife@Home: 6,357,242
Wildlife@Home Watched: 0s
Wildlife@Home Events: 0
Climate Tweets: 240
Images Observed: 0

        
Message 4632 - Posted: 1 Oct 2014, 7:11:48 UTC - in response to Message 4631.

Could we simply have the 3 projects as it was before?:

DNA@Home
SubsetSum@Home
Wildlife@home

Nothing more, nothing less

You have made your point
____________

Profile Steve Hawker*
Send message
Joined: 8 Apr 13
Posts: 134
Combined Credit: 829,896
DNA@Home: 11,932
SubsetSum@Home: 299,708
Wildlife@Home: 518,257
Wildlife@Home Watched: 5,541,577s
Wildlife@Home Events: 2,169
Climate Tweets: 8,645
Images Observed: 55

              
Message 4635 - Posted: 1 Oct 2014, 14:28:25 UTC - in response to Message 4631.
Last modified: 1 Oct 2014, 14:28:46 UTC

Could we simply have the 3 projects as it was before?:

DNA@Home
SubsetSum@Home
Wildlife@home

Nothing more, nothing less



The precedent is set with yoyo and Oproject and to a lesser extent, PrimeGrid.

Wouldn't the easiest way be to just have CSG, deprecate the three sub-projects, and have a banner somewhere that explains the change and why some folks just lost some stats. Eventually everyone will know and the banner can be removed.

I think this is the most consistent approach.

To take care of the sub-project issue, couldn't you have a single MM that counts sub-projects?
____________

Trotador
Send message
Joined: 14 Oct 14
Posts: 4
Combined Credit: 14,003,824
DNA@Home: 1,465,524
SubsetSum@Home: 448,628
Wildlife@Home: 12,089,671
Wildlife@Home Watched: 0s
Wildlife@Home Events: 0
Climate Tweets: 0
Images Observed: 0

      
Message 5015 - Posted: 5 Mar 2015, 6:21:14 UTC

Stats are not being exported for several days now.

Please have a look at it when you can.

Thanks

Travis Desell
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 16 Jan 12
Posts: 1813
Combined Credit: 23,514,257
DNA@Home: 293,563
SubsetSum@Home: 349,212
Wildlife@Home: 22,871,482
Wildlife@Home Watched: 212,926s
Wildlife@Home Events: 51
Climate Tweets: 21
Images Observed: 774

              
Message 5019 - Posted: 7 Mar 2015, 0:51:11 UTC - in response to Message 5015.

Stats are not being exported for several days now.

Please have a look at it when you can.

Thanks


Sorry! Took me awhile to figure this one out. Should be working now. When the disks filled up it screwed up BOINC knowing if it was enabled or not, so the crontasks were failing to run.


Post to thread

Message boards : News : CSG now exporting stats