There's a max WU of 8 with Virtualbox

Message boards : Number crunching : There's a max WU of 8 with Virtualbox

To post messages, you must log in.

Previous · 1 · 2

AuthorMessage
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 2130
Credit: 41,424,155
RAC: 16,102
Message 105120 - Posted: 22 Feb 2022, 8:15:21 UTC - in response to Message 105083.  

My theory . . .

Rings true. I'll bear it in mind when I upgrade the processor on my Intel PC
ID: 105120 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
.clair.

Send message
Joined: 2 Jan 07
Posts: 274
Credit: 26,399,595
RAC: 0
Message 105144 - Posted: 22 Feb 2022, 21:48:17 UTC - in response to Message 105120.  

My theory . . .

Rings true. I'll bear it in mind when I upgrade the processor on my Intel PC

Last year I upgraded my twin xeon system from 24 cpu/theads to the best the motherboard can take e5 2697 48 thread [and now `affordable` on ebay]
it had 32GB in it , I was going to `simply` double it , but having seen what pythons where doing to other systems here I decided to put in a `silly` large amount of RAM in it,
so in went 128GB , I now know it is not a silly lot of memory for it to crunch pythons with as I have seen it using 80GB+ total system use ,
and can only just fit all it`s greedy pythons in.
ID: 105144 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
xii5ku

Send message
Joined: 29 Nov 16
Posts: 22
Credit: 13,889,918
RAC: 4,440
Message 105157 - Posted: 23 Feb 2022, 10:37:08 UTC - in response to Message 105083.  

.clair. wrote:
On my 16cpu system I can only run max 11 pythons together because it has only 32GB memory , 100% available to boinc.
The `properties` of a python want 2.79GB memory , so , 2.79 x 11 = 30.69GB a bit for system use . and its ram is full used
with no other tasks running it uses about 20<21GB of ram the rest boinc calculates has been eaten by pythons , {greedy objects}
Here is a trick which I learned from a teammate, and which I am employed already myself successfully at the Amicable Numbers project (but haven't used at Rosetta@home yet):

If you edit global_prefs_override.xml in the BOINC data directory, you can set an allowed RAM usage of more than 100%. (When computer is in use: <ram_max_used_busy_pct> tag, when not in use: <ram_max_used_idle_pct> tag.) After you edited the file, let the BOINC client read it either by means of boincmgr's menu item "Options" -> "Read local prefs file", or by means of "boinccmd --read_global_prefs_override" on the command line.

Don't set the percentage too high of course, otherwise the computer may end up swapping to disk after all and will be very unresponsive when doing so.
ID: 105157 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
.clair.

Send message
Joined: 2 Jan 07
Posts: 274
Credit: 26,399,595
RAC: 0
Message 105189 - Posted: 24 Feb 2022, 3:48:13 UTC - in response to Message 105157.  
Last modified: 24 Feb 2022, 4:35:00 UTC

.clair. wrote:
On my 16cpu system I can only run max 11 pythons together because it has only 32GB memory , 100% available to boinc.
The `properties` of a python want 2.79GB memory , so , 2.79 x 11 = 30.69GB a bit for system use . and its ram is full used
with no other tasks running it uses about 20<21GB of ram the rest boinc calculates has been eaten by pythons , {greedy objects}
Here is a trick which I learned from a teammate, and which I am employed already myself successfully at the Amicable Numbers project (but haven't used at Rosetta@home yet):

If you edit global_prefs_override.xml in the BOINC data directory, you can set an allowed RAM usage of more than 100%. (When computer is in use: <ram_max_used_busy_pct> tag, when not in use: <ram_max_used_idle_pct> tag.) After you edited the file, let the BOINC client read it either by means of boincmgr's menu item "Options" -> "Read local prefs file", or by means of "boinccmd --read_global_prefs_override" on the command line.

Don't set the percentage too high of course, otherwise the computer may end up swapping to disk after all and will be very unresponsive when doing so.

Thanks xii5ku
THIS WORKS, COZ I JUST BIN AN DONE IT , YAY.....
Now then , boinc mangler gets in a bit of a snit if you try to change any other value in any of the other settings , in the normal "Options" tab.
with a pop up box "validation error" , "value must be between 1<100" , 150% does appear in the value box , but you cant change it , it goes RED , oops . . .
like reducing the number of cpu`s used when you . . . {who??? :-), find I overdun it . . ....
Don't care , me haz new toy to play with so ist just gon an do it in the prefs file wiff nukepad , oops , that's sposed to read notepad
And I now have more cpu`s running than is possible because of greedy pythons, I have checked `properties` and cpu usage to be shure , coz this is fun way the hak the limits
On a system that realy haz got 32GB of RAM I now get this in event log :-
24/02/2022 02:47:03 |  | Reading preferences override file
24/02/2022 02:47:03 |  | Preferences:
24/02/2022 02:47:03 |  | max memory usage when active: 45872.28 MB 
24/02/2022 02:47:03 |  | max memory usage when idle: 45872.28 MB
24/02/2022 02:47:04 |  | Number of usable CPUs has changed from 15 to 14.
24/02/2022 02:47:04 |  | max CPUs used: 14

45872MB is 140% of 32GB fun , I will fine trim it later . That`s a cheeper memory upgrade than Ebay .
For use by experienced boinc nuts only :-), be very carefull with nukepad you could seriously screw up your system , who , me , naaa . . . . . .
I don't know if this will setting survive a reboot ? , I will find out eventualy .
Edit (again) and it was worth staying up till `stupid o`clock` in the morning to do it ,
ID: 105189 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
xii5ku

Send message
Joined: 29 Nov 16
Posts: 22
Credit: 13,889,918
RAC: 4,440
Message 105195 - Posted: 24 Feb 2022, 11:37:56 UTC - in response to Message 105189.  

.clair. wrote:
Now then , boinc mangler gets in a bit of a snit if you try to change any other value in any of the other settings ,
Yes, boincmgr's computing prefs editor wants us to keep the values in certain civilized bounds.

.clair. wrote:
For use by experienced boinc nuts only :-)
I concur.

.clair. wrote:
I don't know if this will setting survive a reboot ?
global_prefs_override.xml is persistent, like cc_config.xml and app_config.xml for example.
ID: 105195 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
.clair.

Send message
Joined: 2 Jan 07
Posts: 274
Credit: 26,399,595
RAC: 0
Message 105199 - Posted: 24 Feb 2022, 16:08:34 UTC - in response to Message 105195.  
Last modified: 24 Feb 2022, 17:03:08 UTC

.clair. wrote:
Now then , boinc mangler gets in a bit of a snit if you try to change any other value in any of the other settings ,
Yes, boincmgr's computing prefs editor wants us to keep the values in certain civilized bounds.
.clair. wrote:
For use by experienced boinc nuts only :-)
I concur.
.clair. wrote:
I don't know if this setting will survive a reboot ?
global_prefs_override.xml is persistent, like cc_config.xml and app_config.xml for example.

Thanks , xii5ku.
#1 - Python tasks have placed some uncivilised demands on the system.
#2 - I am not nuts / mad I just see the world differently . . . .
#3 - Some of the things we are having to do to run python work units are perssistenting me off :-)
ID: 105199 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 · 2

Message boards : Number crunching : There's a max WU of 8 with Virtualbox



©2024 University of Washington
https://www.bakerlab.org