Time to introduce the Quorum System

Message boards : Number crunching : Time to introduce the Quorum System

To post messages, you must log in.

AuthorMessage
PCZ

Send message
Joined: 16 Sep 05
Posts: 26
Credit: 2,024,330
RAC: 0
Message 7097 - Posted: 21 Dec 2005, 21:52:06 UTC

Admins
I know you think it is a waste of resources but it is necessary for the continued good reputation of the project.
Do you want this project to be famous for its science or the ease at which contributers can cheat on credit ?


ID: 7097 · Rating: -3 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile River~~
Avatar

Send message
Joined: 15 Dec 05
Posts: 761
Credit: 285,578
RAC: 0
Message 7112 - Posted: 21 Dec 2005, 23:57:23 UTC - in response to Message 7097.  

Admins
I know you think it is a waste of resources but it is necessary for the continued good reputation of the project.
Do you want this project to be famous for its science or the ease at which contributers can cheat on credit ?




There's a technical issue here. Until very recently WU used a random number seeded (if I understand correctly) from the system clock at startup. That is why the stuck at 1% prob would go away if a result was restarted. It also meant that the project team would not expect two results from the same WU to match at all.

My view is that when the science proper starts (ie when Rosetta is predicting the shape of unknown proteins) then the project will need redundancy; for now, while all the effort is in perfectig the technique there is no advantage to redundnany.

I'd like to make another related point.

A better way for this sort of project to do redundancy is to only double crunch some WU, but not tell the participant which ones you check. Zetsgrid (non-boinc maths DC project) did this, checking the results of newbies intensively, but reducing checks down to 1 in 10 WU once the software trusted you. As you'd lose all past credit if caught cheating it was thought to be a suffucient safeguard.

Now, none of that is possible on BOINC, as the degree of redundancy is set in advance for the WU, and cannot be adjusted between users. Also the 'transparency' where you know who else has your WU means that the one-in-ten idea can't work -- you'd know right away which were being tested.

An example of BOINC being designed with the experience of one project in mind (SETI) which used multiple redundancy, and there being no way to incorporate alternative redundnacy models. Pity, but one of the prices we pay for having an off-the-shelf infrastructure. On the whole the disadvantages are worth putting up with.

River~~
ID: 7112 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Paul D. Buck

Send message
Joined: 17 Sep 05
Posts: 815
Credit: 1,812,737
RAC: 0
Message 7118 - Posted: 22 Dec 2005, 0:53:00 UTC - in response to Message 7112.  

An example of BOINC being designed with the experience of one project in mind (SETI) which used multiple redundancy, and there being no way to incorporate alternative redundnacy models. Pity, but one of the prices we pay for having an off-the-shelf infrastructure. On the whole the disadvantages are worth putting up with.

Well, other models have been thought of ... we just have not the code in place to take advantage of the potentials.

Cjeck is in the mail ...

My thoughts are in a paper in the Wiki ... search on "calibration concepts" ... when we "proof" the FLOPS credit system with the Enhanced SETI application we will be a step closer. THough I think we need some more things in place ...

But, as a fundamental matter, though I despise the inaccuarcy in the credit system, as River said, the errors are not so big we need to throw the bath water out with the baby ...
ID: 7118 · Rating: -1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Jack Schonbrun

Send message
Joined: 1 Nov 05
Posts: 115
Credit: 5,954
RAC: 0
Message 7120 - Posted: 22 Dec 2005, 0:56:19 UTC

We did have an extensive discussion of this on the code release and redundancy thread.

Protecting the integrity of credit is important for all DC projects, and I think some form of searching for users claiming invalid credit will be implemented not too long from now. But I can't give you a time frame. I'd like to be the project that figures out a way to validate credit without any redundancy.
ID: 7120 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Paul D. Buck

Send message
Joined: 17 Sep 05
Posts: 815
Credit: 1,812,737
RAC: 0
Message 7132 - Posted: 22 Dec 2005, 5:59:53 UTC - in response to Message 7120.  

I'd like to be the project that figures out a way to validate credit without any redundancy.

Adding FLOP counting would be a good start ... :)

From there, well, I think with data on FLOPS, CPU time, CPU type, Claims, etc. there should be enough data to figure it out ...

Since we don't have a decent body of FLOP counted work (yet) this will have to wait a bit. But, my look at the SETI@Home Application showed me that the burden of adding the code and actually counting the FLOPS may be less than *I* expected.

I am not sure how they derived their formulations for sure, though I bet that they would be glad to tell us if we asked on the Optimization mailing list ...
ID: 7132 · Rating: -1 · rate: Rate + / Rate - Report as offensive    Reply Quote
FluffyChicken
Avatar

Send message
Joined: 1 Nov 05
Posts: 1260
Credit: 369,635
RAC: 0
Message 7174 - Posted: 22 Dec 2005, 13:54:06 UTC
Last modified: 22 Dec 2005, 13:57:41 UTC

In the mean time, some members of my team team, inculdung myself and most the top teams will be happily distorting the credit system and bringing the league tables of Rosetta and BOINC overall and anything connected with them (like overall flops ratings and throughput estimations) to mean absolutely nothing :-)

Although it is a great publicity idea, "come to Rosetta get higher yourself higher in the Overall BOINC table, feel the advantage!"

I have no choice, the standard boinc client doean't do what I need, so I have to use a 3rd party one. Others because thay use linux and lag behind windows measurments and other because they get a higher score and it's not against rosetta rules so hey why not.


(set's off to figure out how that report whatever you want thing works so I can get to the top quicker >8-)


Team mauisun.org
ID: 7174 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote

Message boards : Number crunching : Time to introduce the Quorum System



©2024 University of Washington
https://www.bakerlab.org