Message boards : Number crunching : How to fake out the new credit system
Previous · 1 · 2 · 3 · Next
Author | Message |
---|---|
Bird-Dog Send message Joined: 15 Dec 05 Posts: 42 Credit: 116,749 RAC: 0 |
Are there any hosts that run a standard client and unmodified xml benchmarks that are getting less granted than claimed? All of mine apart from 1 or 2 are granted less than claimed. Stock everything. (edit) |
Trog Dog Send message Joined: 25 Nov 05 Posts: 129 Credit: 57,345 RAC: 0 |
Does it make any difference if you increase the target run time? |
TestPilot Send message Joined: 13 Jun 06 Posts: 29 Credit: 0 RAC: 0 |
All PowerMACs with the old IBM processors get less than 50%. For example here: Yes! Exactly! So new system grant fair amount of points to computers, irrelevantly of claimed amount of points and benchmark results! Furthermore. At the top of the thread it was suggested that if big enough team over claim points - they will eventually get more points. That is true, but ironically that would not help cheating team. Simply because by over claiming they increasing amount of points not just for themselves, but also for there rivals/competitors! The only effects would be that the project as whole, Rosetta at home will claim slightly more boinc points compare to Einstein@Home / ClimatePrediction.Net... |
Bird-Dog Send message Joined: 15 Dec 05 Posts: 42 Credit: 116,749 RAC: 0 |
I`ll increase it to 2hrs for now when the one its doing is finished. Can`t put it up longer just yet because of other projects. Will increase it more later. |
Trog Dog Send message Joined: 25 Nov 05 Posts: 129 Credit: 57,345 RAC: 0 |
Those mac results really throw a mental spanner in the works. Because we're now using a system that deals wih averages I would've expected the higher scoring (performing) hosts to take a minor drop while the lower scoring hosts take a rise. Intuitively, I would have expected a slight drop in standard AMD's as reported by mnb-fin and Bird-Dog. AMD's were afterall (reportedly) the top of the heap. The problem with this line of reasoning is the PowerMacs, does it really take one approximately 50% longer to create/find/calculate a model as indicated by David Kim's machine as compared to the "average" machine. Are the results reported by David Kim's machine indicative of all powermacs? |
Trog Dog Send message Joined: 25 Nov 05 Posts: 129 Credit: 57,345 RAC: 0 |
True, but if a team or teams (or any group together) set a high cache, overclaim, and have relatively quick machines they should be able to manipulate the system. Whether it's worth it is another matter. The high cache is necessary to get wu's of the same type, and/or to make an influence in a wu batch as early as possible. They will raise everybody elses score initially but the more standard client, and slower hosts return results the average will drop. If they are returning results faster than the averge host then this will help too. |
TestPilot Send message Joined: 13 Jun 06 Posts: 29 Credit: 0 RAC: 0 |
Yes, but to effectively profit from this you need: 1) Overclaim points only for some type of specially selected WU(which is relatively easy to implement) 2) Get thouse WU you are overclaiming on average more often than non cheating computer gets. The problem is 2). Hm... Yes, you can count only WU of that type you are overclaiming! Just return all other types with errors, and wait until server give you thouse WU you really want! So, solution to the topic question found, am I right? |
FluffyChicken Send message Joined: 1 Nov 05 Posts: 1260 Credit: 369,635 RAC: 0 |
Trog Dog, I don't understand how increasing the runtime will alter teh credits ? Since 1x4 result should be the same as 4x1 results, else there is a major flaw in the system? It may 'look' better in the stats listing since the average of your result over 24hrs should be more consistent when shouwn as a bulk credit, than seeing them scored individually. But the overall amount will (should hopefully!) be the same. Team mauisun.org |
STE\/E Send message Joined: 17 Sep 05 Posts: 125 Credit: 4,103,208 RAC: 167 |
I've tried running the WU's at different Time Lengths, 1 hr & 2 hr & 4 hr & 8 hr, the Credits seem to work out the same per hour average no matter what length I run them ... |
Bird-Dog Send message Joined: 15 Dec 05 Posts: 42 Credit: 116,749 RAC: 0 |
Increasing the time has made no differance, same average /hr |
Trog Dog Send message Joined: 25 Nov 05 Posts: 129 Credit: 57,345 RAC: 0 |
That's why the high cache, presuming that wu are relaeased in batches. 10 day cache you would presume a higher cluster of the same wu's than .1 day cache. |
Trog Dog Send message Joined: 25 Nov 05 Posts: 129 Credit: 57,345 RAC: 0 |
Trog Dog, I don't understand how increasing the runtime will alter teh credits ? Since the credit granted is based on the models/decoys found it's just a check of whether the incidence of models/decoys in a wu increases or decreases over time. So if the rate is constant within a wu (ie if you find 1 in a 1 hour wu, then you will find 10 in a 10 hour wu) then you have more chance of influencing the score by returning smaller wu's, because you are returning them quicker you get more bites at the cherry. |
Ingleside Send message Joined: 25 Sep 05 Posts: 107 Credit: 1,514,472 RAC: 0 |
Well, let's say 10% is trying to cheat by claiming too much, how will this effect the average granted credit? Overclaim - increase in average granted credit per model: 5x - 40% 4x - 30% 3x - 20% 2x - 10% 1.5x - 5% 1.1x - 1% Since a constant 2x-overclaim only gives 10% increase, an ocassional high claim will not give any noticeable effect, except if it's one of the very 1st. returned for a specific wu-type. Therefore, someone trying to cheat must basically all the time overclaim... But, this makes it easier to detect server-side, so can example discard any claims that is markedly higher than average, and maybe zeroing a couple of the "worst offenders"... The "weak" spot is at the beginning, so my recommendation is to delay crediting until N% or N models is in... "I make so many mistakes. But then just think of all the mistakes I don't make, although I might." |
zombie67 [MM] Send message Joined: 11 Feb 06 Posts: 316 Credit: 6,621,003 RAC: 0 |
I've tried running the WU's at different Time Lengths, 1 hr & 2 hr & 4 hr & 8 hr, the Credits seem to work out the same per hour average no matter what length I run them ... My team has a stats site that shows your credits over the past 24hours (rolling). When the new credit system came on, it was pointed out that those returning results first, or close to first, can get what they claim (or close to it). So I set my run time to te minimum (1 hr). I was getting consistantly in the high 5k range. Then I tried setting it to 6 hour run time. My numbers increased to the low 7k range. So then I tried 8 hour run time, and they dropped to high 6k range. FWIW, after each change, I waited the runtime + 24 hours to let the rolling 24hr window settle. I am going back to 6 hours to see if it goes back up. It's like target practice in the dark! There is no pattern or reason to the results. Reno, NV Team: SETI.USA |
tralala Send message Joined: 8 Apr 06 Posts: 376 Credit: 581,806 RAC: 0 |
Trog Dog, I don't understand how increasing the runtime will alter teh credits ? <nitpicking> Rosetta spends about 1 minute in initialisation which needs to be done only at startup, so you loose with 1 hour WU about a minute every hour (~1.66 %). ;-) </nitpicking> |
Biggles Send message Joined: 22 Sep 05 Posts: 49 Credit: 102,114 RAC: 0 |
I understand how your exploit works feet1st, but there's a way to limit its effectiveness. BOINC allows for a maximum daily work unit quota, which can be reduced for every failed work unit. If that is enforced that then limits the amount of work units a person can try cheating on. Not perfect by a long shot, but it does limit the amount of times it can be done. And of course if a machine gets to the top host spot and has failed every work unit, it will stand out. |
FluffyChicken Send message Joined: 1 Nov 05 Posts: 1260 Credit: 369,635 RAC: 0 |
Trog Dog, I don't understand how increasing the runtime will alter teh credits ? Doesn't it initialise at the beginning of ever 'run' ? (Since it kind of restarts after each one (a run lenght is actual split up into descrite time lengths) and 1.66% will not be noticable in the variation of decoy lengths anyway ;-) Team mauisun.org |
tralala Send message Joined: 8 Apr 06 Posts: 376 Credit: 581,806 RAC: 0 |
Trog Dog, I don't understand how increasing the runtime will alter teh credits ? No as I said the initialisation is only once performed at startup. Of course those 1.66 % are not noticeable, I just mentioned it for the sake of completeness. |
zombie67 [MM] Send message Joined: 11 Feb 06 Posts: 316 Credit: 6,621,003 RAC: 0 |
|
Feet1st Send message Joined: 30 Dec 05 Posts: 1755 Credit: 4,690,520 RAC: 0 |
I understand how your exploit works feet1st, but there's a way to limit its effectiveness. BOINC allows for a maximum daily work unit quota, which can be reduced for every failed work unit. If that is enforced that then limits the amount of work units a person can try cheating on. Biggles I didn't intentionally abort the WUs. BOINC did it for me for unknown reason. If I kept modifying my file after every benchmark is run by BOINC, I could crunch forward with the claim and not require any extra number of WUs... because, afterall, I'm NOT crunching any faster. My point was the old system is EASY to manipulate. And in various ways, many people did that, and so with relation to actual TeraFLOPS, Rosetta's daily credits issued was inflated as compared to the actual work done. Therefore, it is only to be expected that the daily number would drop under the new system. Teams pulling off... and on, certainly sways the numbers too. The initialization also occurs when Rosetta has been removed from memory. Either by ending BOINC, or by preempting to another project and not keeping in memory. Trog Dog Consider this, if a team reports thousands of models crunched... and happens to both be SENT, AND to RETURN these WUs first with overclaimed credits... if they succeed at ALL of that... then the credit per model granted to EVERYONE downstream from there is also inflated. In essence, for all the effort, they've benefitted everyone (creditwise) as much as themselves. It's a zero-sum game. Add this signature to your EMail: Running Microsoft's "System Idle Process" will never help cure cancer, AIDS nor Alzheimer's. But running Rosetta@home just might! https://boinc.bakerlab.org/rosetta/ |
Message boards :
Number crunching :
How to fake out the new credit system
©2024 University of Washington
https://www.bakerlab.org