Message boards : Number crunching : Time to introduce the Quorum System
Author | Message |
---|---|
PCZ Send message Joined: 16 Sep 05 Posts: 26 Credit: 2,024,330 RAC: 0 |
Admins I know you think it is a waste of resources but it is necessary for the continued good reputation of the project. Do you want this project to be famous for its science or the ease at which contributers can cheat on credit ? |
River~~ Send message Joined: 15 Dec 05 Posts: 761 Credit: 285,578 RAC: 0 |
Admins There's a technical issue here. Until very recently WU used a random number seeded (if I understand correctly) from the system clock at startup. That is why the stuck at 1% prob would go away if a result was restarted. It also meant that the project team would not expect two results from the same WU to match at all. My view is that when the science proper starts (ie when Rosetta is predicting the shape of unknown proteins) then the project will need redundancy; for now, while all the effort is in perfectig the technique there is no advantage to redundnany. I'd like to make another related point. A better way for this sort of project to do redundancy is to only double crunch some WU, but not tell the participant which ones you check. Zetsgrid (non-boinc maths DC project) did this, checking the results of newbies intensively, but reducing checks down to 1 in 10 WU once the software trusted you. As you'd lose all past credit if caught cheating it was thought to be a suffucient safeguard. Now, none of that is possible on BOINC, as the degree of redundancy is set in advance for the WU, and cannot be adjusted between users. Also the 'transparency' where you know who else has your WU means that the one-in-ten idea can't work -- you'd know right away which were being tested. An example of BOINC being designed with the experience of one project in mind (SETI) which used multiple redundancy, and there being no way to incorporate alternative redundnacy models. Pity, but one of the prices we pay for having an off-the-shelf infrastructure. On the whole the disadvantages are worth putting up with. River~~ |
Paul D. Buck Send message Joined: 17 Sep 05 Posts: 815 Credit: 1,812,737 RAC: 0 |
An example of BOINC being designed with the experience of one project in mind (SETI) which used multiple redundancy, and there being no way to incorporate alternative redundnacy models. Pity, but one of the prices we pay for having an off-the-shelf infrastructure. On the whole the disadvantages are worth putting up with. Well, other models have been thought of ... we just have not the code in place to take advantage of the potentials. Cjeck is in the mail ... My thoughts are in a paper in the Wiki ... search on "calibration concepts" ... when we "proof" the FLOPS credit system with the Enhanced SETI application we will be a step closer. THough I think we need some more things in place ... But, as a fundamental matter, though I despise the inaccuarcy in the credit system, as River said, the errors are not so big we need to throw the bath water out with the baby ... |
Jack Schonbrun Send message Joined: 1 Nov 05 Posts: 115 Credit: 5,954 RAC: 0 |
We did have an extensive discussion of this on the code release and redundancy thread. Protecting the integrity of credit is important for all DC projects, and I think some form of searching for users claiming invalid credit will be implemented not too long from now. But I can't give you a time frame. I'd like to be the project that figures out a way to validate credit without any redundancy. |
Paul D. Buck Send message Joined: 17 Sep 05 Posts: 815 Credit: 1,812,737 RAC: 0 |
I'd like to be the project that figures out a way to validate credit without any redundancy. Adding FLOP counting would be a good start ... :) From there, well, I think with data on FLOPS, CPU time, CPU type, Claims, etc. there should be enough data to figure it out ... Since we don't have a decent body of FLOP counted work (yet) this will have to wait a bit. But, my look at the SETI@Home Application showed me that the burden of adding the code and actually counting the FLOPS may be less than *I* expected. I am not sure how they derived their formulations for sure, though I bet that they would be glad to tell us if we asked on the Optimization mailing list ... |
FluffyChicken Send message Joined: 1 Nov 05 Posts: 1260 Credit: 369,635 RAC: 0 |
In the mean time, some members of my team team, inculdung myself and most the top teams will be happily distorting the credit system and bringing the league tables of Rosetta and BOINC overall and anything connected with them (like overall flops ratings and throughput estimations) to mean absolutely nothing :-) Although it is a great publicity idea, "come to Rosetta get higher yourself higher in the Overall BOINC table, feel the advantage!" I have no choice, the standard boinc client doean't do what I need, so I have to use a 3rd party one. Others because thay use linux and lag behind windows measurments and other because they get a higher score and it's not against rosetta rules so hey why not. (set's off to figure out how that report whatever you want thing works so I can get to the top quicker >8-) Team mauisun.org |
Message boards :
Number crunching :
Time to introduce the Quorum System
©2024 University of Washington
https://www.bakerlab.org