Message boards : Number crunching : 1.03 (vbox64) is out for rosetta python projects
Author | Message |
---|---|
Jim1348 Send message Joined: 19 Jan 06 Posts: 881 Credit: 52,257,545 RAC: 0 |
A new version for the rosetta python projects is out for Windows, Linux and Mac. Maybe there will be some work units to go along with it. That would be nice. |
mikey Send message Joined: 5 Jan 06 Posts: 1895 Credit: 9,169,305 RAC: 3,078 |
A new version for the rosetta python projects is out for Windows, Linux and Mac. I agree |
[VENETO] boboviz Send message Joined: 1 Dec 05 Posts: 1994 Credit: 9,623,704 RAC: 7,594 |
Maybe there will be some work units to go along with it. That would be nice. - more python wus - possibility, in the user's profile, to choose app - what are differences between this app and 0.21 in Ralph@home? |
Jim1348 Send message Joined: 19 Jan 06 Posts: 881 Credit: 52,257,545 RAC: 0 |
- possibility, in the user's profile, to choose app Yes, that is my hope. In the meantime, I have set up app_configs for each machine to limit the number of pythons to what its memory can handle. It is even possible that they will run short (or even out) of the regular Rosettas and we will be left with only the pythons. Since they don't tell us, we can believe what we want. |
dcdc Send message Joined: 3 Nov 05 Posts: 1831 Credit: 119,627,225 RAC: 9,274 |
I doubt we'll be out of normal Rosetta units for a while yet because I believe the Robetta server takes requests in from around the world and distributes them on r@h. I wouldn't expect all those users to start using trrosetta yet, but I might well be wrong. Would be good to hear from someone in the project. |
wolfman1360 Send message Joined: 18 Feb 17 Posts: 72 Credit: 18,450,036 RAC: 0 |
Hopefully they don't switch exclusively to vbox - or give users the ability to choose workunit types. There are doubtlessly countless machines that do not have virtual box installed and likely will not do so as they simply sit and crunch with users not checking the forums. How much memory does each WU consume with these new python WUs? Do they finally use SSE/Avx? I don't think I have gotten one yet. thanks |
Jim1348 Send message Joined: 19 Jan 06 Posts: 881 Credit: 52,257,545 RAC: 0 |
How much memory does each WU consume with these new python WUs? You won't believe it. They use 8 GB per work unit. Maybe they have reduced that, but VBox will be the easy part. |
wolfman1360 Send message Joined: 18 Feb 17 Posts: 72 Credit: 18,450,036 RAC: 0 |
How much memory does each WU consume with these new python WUs? 8 GB! Good thing 2 of my Ryzen's have 64, but ...still! That's just a little excessive. Hopefully that comes way down or gets optimized before these go into main production... |
Bryn Mawr Send message Joined: 26 Dec 18 Posts: 393 Credit: 12,110,248 RAC: 4,484 |
How much memory does each WU consume with these new python WUs? If these become the standard then I will have to, sadly, abandon Rosetta. |
mikey Send message Joined: 5 Jan 06 Posts: 1895 Credit: 9,169,305 RAC: 3,078 |
How much memory does each WU consume with these new python WUs? WHY? You can always run 1 task at a time? |
Bryn Mawr Send message Joined: 26 Dec 18 Posts: 393 Credit: 12,110,248 RAC: 4,484 |
How much memory does each WU consume with these new python WUs? So run 1, possibly 2, tasks and leave the other 23/22 threads idle on each machine? No thanks, Rosetta is one of 4 projects I contribute to and I would have to sacrifice it to allow the other 3 to continue running. |
Jim1348 Send message Joined: 19 Jan 06 Posts: 881 Credit: 52,257,545 RAC: 0 |
So run 1, possibly 2, tasks and leave the other 23/22 threads idle on each machine? You use an app_config.xml file to limit the Pythons to the number you can run. https://boinc.bakerlab.org/rosetta/forum_thread.php?id=14448&postid=102001#102001 Then you can use the other cores to run whatever else you want (which may be the ordinary Rosettas). It is a good way to use up all the memory you have. |
mikey Send message Joined: 5 Jan 06 Posts: 1895 Credit: 9,169,305 RAC: 3,078 |
So run 1, possibly 2, tasks and leave the other 23/22 threads idle on each machine? In that app_config file you don't even need the name of the app, you can leave that line out and as long as it's in the Project folder you can run x numbers of units at one time from that project. I often use to limit my crunching when new apps come out from the different projects, limiting which apps I get in the different venues when possible, and run ie 3 tasks from project a, 3 tasks from project b and then 6 tasks from project c still leaving 2 cores free for gpu crunching and surfing. It can mean a little hands on to get tasks from some projects but suspending the ones I dod not need tasks from and then resuming them after I get the needed tasks means things run pretty smoothly. <app_config> <app> <max_concurrent>3</max_concurrent> </app> </app_config> It does NOT work from projects that do not let you pick which type of task you want to run in the venue setting, ie Rosetta, but if you don't care what kind of tasks you want to crunch it works fine. |
Jim1348 Send message Joined: 19 Jan 06 Posts: 881 Credit: 52,257,545 RAC: 0 |
In that app_config file you don't even need the name of the app, you can leave that line out and as long as it's in the Project folder you can run x numbers of units at one time from that project. Yes, but then you limit all the tasks, even the normal Rosetta ones. If you use the name, you can limit the Python Rosettas while not restricting the normal Rosettas. If you have limited memory, it would be best to use a non-Rosetta "other" project. WCG/MCM works well, though it sometimes gives me scheduling problems and downloads too many. TN-Grid is nice and reliable with good science and minimal memory requirements at 57 MB per work unit. http://gene.disi.unitn.it/test/index.php And if you want covid research, there is SiDock, using 173 MB per work unit (all on LInux). |
Bryn Mawr Send message Joined: 26 Dec 18 Posts: 393 Credit: 12,110,248 RAC: 4,484 |
So run 1, possibly 2, tasks and leave the other 23/22 threads idle on each machine? If 2 tasks use up all of the memory on the machine then there is no memory for the other cores to run anything. As I said, if the new tasks become the norm (I.e. Rosetta no longer provides the current type of task) then I will stop running Rosetta. |
[VENETO] boboviz Send message Joined: 1 Dec 05 Posts: 1994 Credit: 9,623,704 RAC: 7,594 |
Calm down, guys. I think it's just the beginning of the "python project" and that there is a lot of work to do on the code. |
wolfman1360 Send message Joined: 18 Feb 17 Posts: 72 Credit: 18,450,036 RAC: 0 |
One upside to these - at least from what I can see - they take a lot less time to run, maybe an hour. My two larger processors - 2 Ryzen 7-3700's. Sidock seems to do okay with them - nice and small requirements - but sometimes I run into scheduling issues. Even my 4770 has problems and the scheduler can't keep up at times. I should check out tn grid again. I gave up pretty hard on LHC at home which is a shame as the science was very neat, but ran into constant issues, I do not have time to fiddle with things these days, given it is summer and the beach calls. These Ryzen's are doing so much work versus the e5-2680 I had going in here and my air conditioner is kicking in so much less. Add to that - the incredible amount of hot temperatures and it's only June, best purchase I have made. How do the 5000 series from AMD (Zen 3) do on this project? I couldn't find one at a decent price, the 5600 was more expensive than this 3700. I might snatch up yet another 3000 series - maybe a 3600. |
Jim1348 Send message Joined: 19 Jan 06 Posts: 881 Credit: 52,257,545 RAC: 0 |
I might snatch up yet another 3000 series - maybe a 3600. I like the 3600 a lot, though I have 3900X and 3950X also. But the cache sometimes works better on the 3600 than on the larger chips, depending on how the project uses it, and they are easier to cool. And the price was right when I bought them. Anything is more now, but I am going to wait at least until the end of the year for the 5000 series. |
Grant (SSSF) Send message Joined: 28 Mar 20 Posts: 1682 Credit: 17,854,150 RAC: 18,215 |
One upside to these - at least from what I can see - they take a lot less time to run, maybe an hour.How long they run for is set by the project. When the final version is released i'd expect their runtime to be the project's default of 8 hours. Since they're still developing the application, they would want the results back sooner, hence the shorter runtime. Grant Darwin NT |
[VENETO] boboviz Send message Joined: 1 Dec 05 Posts: 1994 Credit: 9,623,704 RAC: 7,594 |
A little batch of rosetta python on Ralph. |
Message boards :
Number crunching :
1.03 (vbox64) is out for rosetta python projects
©2024 University of Washington
https://www.bakerlab.org