is 512MB RAM really needed?

Message boards : Number crunching : is 512MB RAM really needed?

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Profile River~~
Avatar

Send message
Joined: 15 Dec 05
Posts: 761
Credit: 285,578
RAC: 0
Message 8015 - Posted: 30 Dec 2005, 21:37:48 UTC

OK - first the disclaimer.

The project team advise, just one mouse click from the home page, that you don't run Rosetta if you have less than 512MB RAM. This means the official position is just that - try it with anything less and if it does not work there is no committment to support you. If it goes wrong, you can ask for help but there may not be any. If there is it will be from volunteers not from the project.

---

However, it seems odd that the limits are the same for such varying operating systems as WinXP, Win2k, Linux, and Mac OS X. And apparently no extra needed for HT and twin-cpu boxes.

WinXP typically needs a lot more RAM than Win2k, and just from that I'd hope that Win2k would need no more than maybe 320MB to 384MB.

Twin-processoor boxes - how much extra RAM should you allow for each exra cpu (incuding HT 'virtual' cpus). So if XP is good for twin-running on an HT box with only 512MB, that brings the win-2k memory needs even lower on a single cpu box - or so I'd imagine.


Win98 is not even supported - but it takes a smaller chunk out of RAM for the OS, so the twin 'offences' of Win98 and (say) 320MB may 'cancel out' at least as far as memory needs.

Linux - surely it matters if you are running a GUI or not? The client itself runs from the command line, so if a 512MB Linux box runs well with KDE installed and both Xfree and KDE running, then surely a comman-line-only linux would run in maybe 256MB? Or 192MB?

So the first point is that the guidelines would be more credible if they posted differing figures for different operating systems and numbers of processors (including HT virtual procs).

The second point is that about half of my boxes are below the official spec, yet all my boxes can turn in Rosetta wu. When I joined Rosetta -- forgive me but I didn't stop to read the spec. 10K cobblestones later I discover that maybe those boxes shouldn't have come in the first place.

Have I unwittingly made the problems of the last few weeks worse for myself by running sub-spec equipment?

Or is the project being overly cautious to leave space for error? To leave space for future increases in code size without having to send donors away?

So I am asking other participants: do you have experience of running with less than 512 RAM? Good experience, bad? or mixed?

And I am asking the project team - in that spare ime you will have so much of ;-) could you adjust the official advice at least to include 2cpu boxes and command-line Linux; and have some differential between XP and 2k.

Because if 512 is always enough, 512 will not always be needed. If 512 is always needed, 512 will not always be enough. Or so it seems to me.

River~~

ID: 8015 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Housing and Food Services

Send message
Joined: 1 Jul 05
Posts: 85
Credit: 155,098,531
RAC: 0
Message 8018 - Posted: 30 Dec 2005, 22:44:55 UTC - in response to Message 8015.  
Last modified: 30 Dec 2005, 22:45:17 UTC

I have probably half a dozen running with 128, and another 30-40 with 256. I haven't seen any problems other than the issues that aren't memory related (bad random numbers, default_205, etc). Users have not made any comments that machines with 128 are running any slower than they were before rosetta.

So, I'd say it's safe to run with less memory as long as you know you'll be using your disk more often and that *could* shorten the life of the drive.

With that said, the 512 requirement has usually been tossed around for work units down the road when they go to bigger, more complex proteins.

-Ethan


ID: 8018 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Lee Carre

Send message
Joined: 6 Oct 05
Posts: 96
Credit: 79,331
RAC: 0
Message 8031 - Posted: 31 Dec 2005, 2:24:18 UTC
Last modified: 31 Dec 2005, 2:26:56 UTC

i have been sucessfully running rosetta on a P3-500 with 384MB from when i joined, that box chugs along quite happily, and is used for a veriety of things, some that use a fair bit of memory, like playing hi-def music files, or running firefox with a hundred or so tabs open with no problems at all

I always thought 512 was a too high, especially with the improved app a while back that used even less memory, i now rarely see a rosetta app use above 100MB, typically between 50 and 90

but as you say, maybe they're just covering themselves for later on when they get more complex apps and proteins to deal with
ID: 8031 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [B@H] Ray
Avatar

Send message
Joined: 20 Sep 05
Posts: 118
Credit: 100,251
RAC: 0
Message 8127 - Posted: 1 Jan 2006, 19:54:45 UTC

I wass running Rosetta on Win 98 with no problem for a while.

(512 megs of ram)
I started with 256 megs, but increased that for things other than the projects. When I increased the ram to 512 megs the times went down quite a bit, only a little improvement going to 1 gig of ram.

Some units do use more than 256 megs of ram, and will use the swap file for extra memory. This is bad for the hard drive and will cause it to ware out faster.

Ray


Pizza@Home Rays Place Rays place Forums
ID: 8127 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Lee Carre

Send message
Joined: 6 Oct 05
Posts: 96
Credit: 79,331
RAC: 0
Message 8159 - Posted: 2 Jan 2006, 4:45:41 UTC - in response to Message 8127.  
Last modified: 2 Jan 2006, 4:45:53 UTC

Some units do use more than 256 megs of ram, and will use the swap file for extra memory. This is bad for the hard drive and will cause it to ware out faster.

not to mention being a killer for system responciveness ;) lol
ID: 8159 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Grutte Pier [Wa Oars]~GP500

Send message
Joined: 30 Nov 05
Posts: 14
Credit: 432,089
RAC: 0
Message 8174 - Posted: 2 Jan 2006, 12:56:47 UTC

i would like if rosetta needed a bit less RAM.
How is the optimising of this improving btw.

don't read much about more effecient coding (CPU/RAM)
ID: 8174 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
thom217

Send message
Joined: 21 Oct 05
Posts: 5
Credit: 0
RAC: 0
Message 8184 - Posted: 2 Jan 2006, 15:37:09 UTC

Is it possible to upgrade the RAM in your machine. Prices have gone down a lot. The last time I checked, there was 1 GB DDRAM chip selling for $90 Canadian.
ID: 8184 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Plum Ugly

Send message
Joined: 3 Nov 05
Posts: 24
Credit: 2,005,763
RAC: 0
Message 8191 - Posted: 2 Jan 2006, 18:03:35 UTC

I have one,ctl300 that is a k6-300/256 ram that runs on Xandros/linux box.Not super fast but it runs with no problems.
ID: 8191 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Lee Carre

Send message
Joined: 6 Oct 05
Posts: 96
Credit: 79,331
RAC: 0
Message 8193 - Posted: 2 Jan 2006, 18:27:51 UTC - in response to Message 8184.  

Is it possible to upgrade the RAM in your machine.

if that's a genuine question then yes, it's entirely possible, you should be able to add new modules to empty slots, or replace existing modules if you wish, just make sure your mothermoard supports the type and size of module you're going to buy, otherwise it won't work
ID: 8193 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Grutte Pier [Wa Oars]~GP500

Send message
Joined: 30 Nov 05
Posts: 14
Credit: 432,089
RAC: 0
Message 8198 - Posted: 2 Jan 2006, 19:02:48 UTC - in response to Message 8184.  
Last modified: 2 Jan 2006, 19:03:12 UTC

Is it possible to upgrade the RAM in your machine. Prices have gone down a lot. The last time I checked, there was 1 GB DDRAM chip selling for $90 Canadian.


And if not genuine, lower ram needs is just nicer for the simple reason that more people will run it.
ID: 8198 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
David Baker
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 17 Sep 05
Posts: 705
Credit: 559,847
RAC: 0
Message 8214 - Posted: 2 Jan 2006, 22:22:50 UTC - in response to Message 8174.  

i would like if rosetta needed a bit less RAM.
How is the optimising of this improving btw.

don't read much about more effecient coding (CPU/RAM)



We've been working on this, but we are running up against some hard limits that will be hard to overcome. Most of the jobs going out these days should be comfortably under 200 MB. the 512MB recommendation came from the very first batch of BOINC runs back in september which involved somewhat larger proteins (which require more memory).
ID: 8214 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Lee Carre

Send message
Joined: 6 Oct 05
Posts: 96
Credit: 79,331
RAC: 0
Message 8218 - Posted: 2 Jan 2006, 23:22:33 UTC - in response to Message 8214.  

We've been working on this, but we are running up against some hard limits that will be hard to overcome. Most of the jobs going out these days should be comfortably under 200 MB. the 512MB recommendation came from the very first batch of BOINC runs back in september which involved somewhat larger proteins (which require more memory).

maybe put a small disclaimer on the system requirements page stating that 512 is the "official" recommendation, but is not a "hard" requirement, and participants may have sucess running rosetta on hosts with much lower RAM capacities (but they do so at their own risk?)
ID: 8218 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
thom217

Send message
Joined: 21 Oct 05
Posts: 5
Credit: 0
RAC: 0
Message 8220 - Posted: 2 Jan 2006, 23:30:11 UTC - in response to Message 8184.  

Is it possible to upgrade the RAM in your machine. Prices have gone down a lot. The last time I checked, there was 1 GB DDRAM chip selling for $90 Canadian.



Yes, actually I meant it as a question :). No hard feelings everyone. I have an older machine with RAM < 512 MB that I could use to run Rosetta on if it requires less RAM.
ID: 8220 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Lee Carre

Send message
Joined: 6 Oct 05
Posts: 96
Credit: 79,331
RAC: 0
Message 8239 - Posted: 3 Jan 2006, 2:30:43 UTC - in response to Message 8220.  
Last modified: 3 Jan 2006, 2:34:14 UTC

Is it possible to upgrade the RAM in your machine. Prices have gone down a lot. The last time I checked, there was 1 GB DDRAM chip selling for $90 Canadian.



Yes, actually I meant it as a question :). No hard feelings everyone. I have an older machine with RAM < 512 MB that I could use to run Rosetta on if it requires less RAM.

yes, entirely, i've upgraded pentium3 machines to have a decent amount a while back, only 1 has 512 thou, all the others run rosetta just fine though, no issues with memory size etc.
so first thing, try it before you spend any money, if it's really bad (uses the hard drive lots) or you need more ram anyway, then get some more, memory for old machines is really cheap now, but depending on what the machine does, or what it's used for, you shouldn't need that much, more memory is never a bad thing, but you start seeing less benifit the more you have, because you have excess, and other things will limit system performance more, typically your hard drive or motherboard interfaces
ID: 8239 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Webmaster Yoda
Avatar

Send message
Joined: 17 Sep 05
Posts: 161
Credit: 162,253
RAC: 0
Message 8244 - Posted: 3 Jan 2006, 4:03:13 UTC - in response to Message 8214.  
Last modified: 3 Jan 2006, 4:13:43 UTC

We've been working on this, but we are running up against some hard limits that will be hard to overcome. Most of the jobs going out these days should be comfortably under 200 MB.


Maybe I'm over-simplifying and maybe there's no solution but I just get the feeling that the memory use and lack of CPU checkpoints and saves are related.

With some of the latest batch of work units taking 9 hours* on a 3.4GHz Pentium 4, there's a lot of crunching being lost when I shut down PCs at night, reboot them (for other, unrelated issues), etc. On slower machines it can be many hours between saves.

Maybe if Rosetta could checkpoint and save to disk more often it would not need to keep as much data in (virtual or real) RAM?

* NO_RANDOM_WTS_OR_FRAGS_1ogw_223_61_2 took just over 9 hours - I wouldn't like to run that on a 500MHz machine!
*** Join BOINC@Australia today ***
ID: 8244 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Paul D. Buck

Send message
Joined: 17 Sep 05
Posts: 815
Credit: 1,812,737
RAC: 0
Message 8257 - Posted: 3 Jan 2006, 8:52:25 UTC

Just one other point, *USUALLY* optimizing for speed and memory footprint are reciprocal operations. Improve one usually costs the other. And to be honest, my choice would be for speed ...

I want to get more done ...

If that computer is that limited, perhaps another project is a better candidate.
ID: 8257 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile dcdc

Send message
Joined: 3 Nov 05
Posts: 1832
Credit: 119,821,902
RAC: 15,180
Message 8269 - Posted: 3 Jan 2006, 13:20:49 UTC
Last modified: 3 Jan 2006, 13:23:38 UTC

does rosetta allocate work units on expected ram/cpu useage? if not, would it be practical to implement an allocation system based on the system's capacity?

I assume it would be a case of ranking work units by expected time to process and expected ram utilisation (or do these scale together?), and then farming out the highest ranking job appropriate for the computer that is requesting work.

I would expect that it's not possible to accurately predict the amount of processing required for individual jobs, but there does seem to be some pattern, and I assume it's mainly down to a combination of the resolution of the modelling, and the size of the protein model.

I've got access to five P3-500/600s with 256/384MB RAM but I don't want to install rosie in a way that will adversely affect the users. I'm going to install on one as a test anyway, but it might be a case of running as a screensaver only at the moment.

Are there recommended settings for running with minimal RAM? I know there's a problem when not leaving rosie in memory at the moment

cheers
Danny
ID: 8269 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
David Baker
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 17 Sep 05
Posts: 705
Credit: 559,847
RAC: 0
Message 8274 - Posted: 3 Jan 2006, 15:29:44 UTC - in response to Message 8244.  

We've been working on this, but we are running up against some hard limits that will be hard to overcome. Most of the jobs going out these days should be comfortably under 200 MB.


Maybe I'm over-simplifying and maybe there's no solution but I just get the feeling that the memory use and lack of CPU checkpoints and saves are related.

With some of the latest batch of work units taking 9 hours* on a 3.4GHz Pentium 4, there's a lot of crunching being lost when I shut down PCs at night, reboot them (for other, unrelated issues), etc. On slower machines it can be many hours between saves.

Maybe if Rosetta could checkpoint and save to disk more often it would not need to keep as much data in (virtual or real) RAM?

* NO_RANDOM_WTS_OR_FRAGS_1ogw_223_61_2 took just over 9 hours - I wouldn't like to run that on a 500MHz machine!



The balance is between the amount of network traffic and the length of the jobs. Each WU is doing ten independent folding trajectories. If we cut this down to two, for example, jobs would run 5 fold faster, but the traffic would go up five fold. Once we have more hardware in place we should be able to deal with more traffic. But the work shouldn't be completely lost--if you have already written out five structures when you turn off your computer off, the next time you turn it on it should only have to make five more (I need to check on this).
ID: 8274 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
David Baker
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 17 Sep 05
Posts: 705
Credit: 559,847
RAC: 0
Message 8275 - Posted: 3 Jan 2006, 15:30:55 UTC - in response to Message 8269.  

does rosetta allocate work units on expected ram/cpu useage? if not, would it be practical to implement an allocation system based on the system's capacity?

I assume it would be a case of ranking work units by expected time to process and expected ram utilisation (or do these scale together?), and then farming out the highest ranking job appropriate for the computer that is requesting work.

I would expect that it's not possible to accurately predict the amount of processing required for individual jobs, but there does seem to be some pattern, and I assume it's mainly down to a combination of the resolution of the modelling, and the size of the protein model.

I've got access to five P3-500/600s with 256/384MB RAM but I don't want to install rosie in a way that will adversely affect the users. I'm going to install on one as a test anyway, but it might be a case of running as a screensaver only at the moment.

Are there recommended settings for running with minimal RAM? I know there's a problem when not leaving rosie in memory at the moment

cheers
Danny



We would very much like to be able to do this, but it doesn't seem that the BOINC infrastructure allows this.

ID: 8275 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Webmaster Yoda
Avatar

Send message
Joined: 17 Sep 05
Posts: 161
Credit: 162,253
RAC: 0
Message 8277 - Posted: 3 Jan 2006, 16:10:08 UTC - in response to Message 8274.  
Last modified: 3 Jan 2006, 16:14:17 UTC

But the work shouldn't be completely lost--if you have already written out five structures when you turn off your computer off, the next time you turn it on it should only have to make five more (I need to check on this).


I'm aware of that. The issue is that, especially on the longer WUs, each of these structures can take a long time to do, so a lot of calculations are wasted when computers are switched off.

It is midnight here and I'm about to switch the computers off. One of them is showing 1% complete after 26 minutes crunching (on an Athlon XP 3000+, a reasonably quick machine). Unless it advances to 10% in the next few minutes, that half hour of crunching will be wasted as we go back to zero in the morning.
With a slow machine (say the minimum, 500MHz PC), the steps could be hours apart.

Most other projects are able to show more even progress and save more often. I'd like to see Rosetta do that too, if at all possible.

*** Join BOINC@Australia today ***
ID: 8277 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
1 · 2 · Next

Message boards : Number crunching : is 512MB RAM really needed?



©2024 University of Washington
https://www.bakerlab.org