Flag to opt-in to run big (200+MB memory) WUs?

Message boards : Number crunching : Flag to opt-in to run big (200+MB memory) WUs?

To post messages, you must log in.

AuthorMessage
Profile Dimitris Hatzopoulos

Send message
Joined: 5 Jan 06
Posts: 336
Credit: 80,939
RAC: 0
Message 12720 - Posted: 27 Mar 2006, 12:39:36 UTC

I just noticed a pretty big (many amino-acids) protein, named 7521_1_fullatom_relax_... which has a "working set" (i.e. memory requirement) of 200MB and a virtual size of 646MB.

I also understand that more big WUs will come in the future, as part of the vaccine and cancer studies within Rosetta@home

I wonder if it would be possible for BOINC to either:

1/ have BOINC feeder only send bigger WUs to hosts with e.g. >768MB

or

2/ let users opt-in to run bigger WUs using a project-specific BOINC setting, like the "run-WU-for-X-hours"


Best UFO Resources
Wikipedia R@h
How-To: Join Distributed Computing projects that benefit humanity
ID: 12720 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Keck_Komputers
Avatar

Send message
Joined: 17 Sep 05
Posts: 211
Credit: 4,246,150
RAC: 0
Message 12726 - Posted: 27 Mar 2006, 17:02:59 UTC

Option one is possible by customising the memory requirement as the tasks are created.

No one has gotten option 2 working yet even though there are a couple of projects that could use it.
BOINC WIKI

BOINCing since 2002/12/8
ID: 12726 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Dimitris Hatzopoulos

Send message
Joined: 5 Jan 06
Posts: 336
Credit: 80,939
RAC: 0
Message 12804 - Posted: 29 Mar 2006, 23:53:05 UTC
Last modified: 30 Mar 2006, 0:02:20 UTC

I'd like to re-assert the issue of "opting-in" to run bigger (>100MB memory) WUs for Rosetta. I don't think the impact on donor userbase should be understimated, i.e. some people will stop crunching (see misc unit err thread).

Rom wrote
That error code useally means the machine ran out of memory during the execution of the workunit. Since you only have 512MB of RAM and one instance of Rosetta can use up to 250MB of Ram, I would recommend turning off HT.


...
Think they need to "tweak" this to state: "PER PROCESS". In the meantime, will go back to crunching for other projects. (edit) All the other projects I crunch for don't have any issues with regards to only having 512Mb of memory...!


Most BOINC projects (except CPDN) require 1/10th or even less memory than Rosetta currently does: most Rosetta WU's need 70-120MB and bigger ones 250MB (and 650MB v.size) compared to e.g. Einstein 12-15MB, SIMAP 5-7MB etc

This BigWU setting would ideally be tunable PER HOST, otherwise per separate home/work/school preferences.

IMHO the best method is how Folding@home does it:


The configuration setting Accept WUs>5MB (also known as the "BigWU" setting) allows for large WUs (large in either download size, upload size, or RAM requirements). Note that these work units are the most demanding work units in download and/or RAM. In the console client, this is set through the -config flag. You will be asked "Allow receipt of work unit assignments and return of work results greater than 5MB in size (such units may have large memory demands)". In the GUI client, this is set through the config panel. We do not recommend the BigWU setting for clients with modems, due to the large filesize involved (and greater potential for transmission problems and WU loss).

Combining advanced methods and BigWU: this combination unlocks the most demanding work units, including QMD and very large RAM and/or download WUs. We do not suggest this setting for a machine which is not readily administered, as this setting is the most demanding and would likely require more donor intervention than any other setting. To compensate for this, these WUs have a large bonus. The donor should monitor the client regularly and report unusual situations to the folding community forum (http://forum.folding-community.org). THIS COMBINATION SHOULD BE ONLY USED WITH EXTREME CAUTION AND FAMILIARITY WITH FAH -- THERE WILL BE HUGE DEMANDS ON THE CLIENT MACHINE, INCLUDING (BUT NOT LIMITED TO) RAM, HARD DRIVE, NETWORK CONNECTION. Bonus points are awarded due to these increased demands.


source: F@h settings


Best UFO Resources
Wikipedia R@h
How-To: Join Distributed Computing projects that benefit humanity
ID: 12804 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile River~~
Avatar

Send message
Joined: 15 Dec 05
Posts: 761
Credit: 285,578
RAC: 0
Message 12872 - Posted: 31 Mar 2006, 9:26:47 UTC

This is actually a very different request to the recent one about the length of WU.

At least, it looks similar to the user, but is very different in terms of the implementation.

The recent mod was to the application code, and affects how the application decides whether it is finished. As such it was totally within the Rosetta team's domain to make this change, and I am glad they did.

In contrast, other opt in flags would need to affect which work was assigned to the computer before it even was issued. This would need mods to the BOINC code in the server as well as mods to the preferences database.

As well as the flag suggested here, I can think of at least four other flags that might be useful, and if any of them is implemented it seems to me to make sense for the BOINC folks to provide the hooks for all of them at the same time - all of them have the same basic idea - that the user sets a flag, and the project team give the scheduler enough info to know which WU are allowed and which excluded by that flag.

I have started a separate thread to discuss whether we feel there is enough demand for such changes to be worth asking the BOINC people to do this, see here
ID: 12872 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote

Message boards : Number crunching : Flag to opt-in to run big (200+MB memory) WUs?



©2024 University of Washington
https://www.bakerlab.org