Problems and Technical Issues with Rosetta@home

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home

To post messages, you must log in.

Previous · 1 . . . 34 · 35 · 36 · 37 · 38 · 39 · 40 . . . 55 · Next

AuthorMessage
Profile David E K
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 1 Jul 05
Posts: 1480
Credit: 4,334,829
RAC: 0
Message 77226 - Posted: 3 Aug 2014, 1:53:02 UTC

Yep, I'm currently optimizing the number of connections on all our servers. Looks like they can keep up without too much load/memory usage so far. These servers are pretty old and I'm sure we'll upgrade soon hopefully.
ID: 77226 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Polian
Avatar

Send message
Joined: 21 Sep 05
Posts: 152
Credit: 10,141,266
RAC: 0
Message 77227 - Posted: 3 Aug 2014, 1:53:56 UTC

ID: 77227 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Miklos M

Send message
Joined: 8 Dec 13
Posts: 29
Credit: 5,277,251
RAC: 0
Message 77228 - Posted: 3 Aug 2014, 1:54:36 UTC

All waiting uploads just went through.
ID: 77228 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile David E K
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 1 Jul 05
Posts: 1480
Credit: 4,334,829
RAC: 0
Message 77229 - Posted: 3 Aug 2014, 1:57:45 UTC

Hmm, I also suspected this but the IP's from the logs were coming from various places. Maybe I'll have to disable new users for now until we figure things out.
ID: 77229 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Polian
Avatar

Send message
Joined: 21 Sep 05
Posts: 152
Credit: 10,141,266
RAC: 0
Message 77230 - Posted: 3 Aug 2014, 2:03:26 UTC - in response to Message 77229.  

Hmm, I also suspected this but the IP's from the logs were coming from various places. Maybe I'll have to disable new users for now until we figure things out.


If it's coming from all over - DDoS then... oh dear. This is conjecture, but it's plausible that someone wrote malicious code to install BOINC as a background service, create a new user and attach it to Rosetta.
ID: 77230 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 2538
Credit: 47,093,569
RAC: 1,341
Message 77231 - Posted: 3 Aug 2014, 2:11:56 UTC - in response to Message 77227.  

ID: 77231 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 2538
Credit: 47,093,569
RAC: 1,341
Message 77232 - Posted: 3 Aug 2014, 2:17:45 UTC - in response to Message 77227.  

Looks like a DoS attack to me, to be honest:

I just picked a random new user ID, 680000 and went up by one from there

Same for 780000...
ID: 77232 · Rating: 0 · rate: Rate + / Rate - Report as offensive
krypton
Volunteer moderator
Project developer
Project scientist

Send message
Joined: 16 Nov 11
Posts: 108
Credit: 2,164,309
RAC: 0
Message 77233 - Posted: 3 Aug 2014, 3:22:04 UTC
Last modified: 3 Aug 2014, 3:22:59 UTC

Yeah, crazy... it's up to
https://boinc.bakerlab.org/rosetta/show_user.php?userid=814655
ID: 77233 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Murasaki
Avatar

Send message
Joined: 20 Apr 06
Posts: 303
Credit: 511,418
RAC: 0
Message 77234 - Posted: 3 Aug 2014, 4:08:40 UTC

The BOINC stats user graphs make interesting reading. Less than 100 new users a day in June and then thousands a day in early and late July.

I think the BOINC stats "new users" is pulled from the stats on the main page. Does "new users" mean new accounts or people claiming their first credit?

If it is the latter then it is not just simply a case of someone trying to shut Rosetta down with a DDoS. Perhaps a "benevolent" hacker signing up their bot net to help the Rosetta science?

On the other hand, the Epstein-Barr news story was picked up by a lot of media outlets in late June/early July. Is this a genuine reaction from a public eager to help speed along a cure?
ID: 77234 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Polian
Avatar

Send message
Joined: 21 Sep 05
Posts: 152
Credit: 10,141,266
RAC: 0
Message 77235 - Posted: 3 Aug 2014, 4:26:22 UTC - in response to Message 77234.  

Is this a genuine reaction from a public eager to help speed along a cure?


Don't think so; if you look at those User IDs, they all seem to be randomly generated - ce followed by a series of numbers.

ID: 77235 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Murasaki
Avatar

Send message
Joined: 20 Apr 06
Posts: 303
Credit: 511,418
RAC: 0
Message 77236 - Posted: 3 Aug 2014, 4:28:03 UTC
Last modified: 3 Aug 2014, 4:31:49 UTC

Looking at the third graph shows Rosetta jumped from under 27,000 active users in June to over 37,000 active users at the start of August.


Edit:

Don't think so; if you look at those User IDs, they all seem to be randomly generated - ce followed by a series of numbers.


Perhaps two separate issues happening at the same time? A sudden surge in active users and a DDoS attack both brought on by recent media attention? I'm not an expert on IT though, so these are just random speculations.
ID: 77236 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile David E K
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 1 Jul 05
Posts: 1480
Credit: 4,334,829
RAC: 0
Message 77237 - Posted: 3 Aug 2014, 5:07:30 UTC

I was told by Matthew Blumberg at Gridrepublic that the new users are real crunchers and that they "started a new marketing campaign via charityengine.com." So I re-enabled the account creation for these users. Our servers may get sluggish again but hopefully things will settle down as the new user rates decrease. And hopefully optimizing the connections on our servers will help. In the future, we hope to get more servers.

This issue coincided with the annual RosettaCon meeting (so most of the Baker lab members were out of town), a final ramp up in CASP targets, and me going on a family camping vacation where I had no phone reception. Normally, I would have been able to react faster to debug and help diffuse the situation.

I am sorry for any inconvenience and the fact that it took a few days to finally make some progress.
ID: 77237 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Polian
Avatar

Send message
Joined: 21 Sep 05
Posts: 152
Credit: 10,141,266
RAC: 0
Message 77238 - Posted: 3 Aug 2014, 5:39:06 UTC

Wow, real people! Neat... and Charity Engine would explain the "ce"
ID: 77238 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 2538
Credit: 47,093,569
RAC: 1,341
Message 77239 - Posted: 3 Aug 2014, 6:05:05 UTC - in response to Message 77237.  

I was told by Matthew Blumberg at Gridrepublic that the new users are real crunchers and that they "started a new marketing campaign via charityengine.com." So I re-enabled the account creation for these users. Our servers may get sluggish again but hopefully things will settle down as the new user rates decrease. And hopefully optimizing the connections on our servers will help. In the future, we hope to get more servers.


Charity Engine - About Us

Charities - Research

Oh! I'd never heard of it before, but Matt Blumberg is part of their Tech Team, so I guess he would know.
ID: 77239 · Rating: 0 · rate: Rate + / Rate - Report as offensive
AtHomer
Avatar

Send message
Joined: 26 Jan 10
Posts: 13
Credit: 7,180,613
RAC: 48
Message 77242 - Posted: 3 Aug 2014, 7:03:54 UTC - in response to Message 77239.  

I was told by Matthew Blumberg at Gridrepublic that the new users are real crunchers and that they "started a new marketing campaign via charityengine.com." So I re-enabled the account creation for these users. Our servers may get sluggish again but hopefully things will settle down as the new user rates decrease. And hopefully optimizing the connections on our servers will help. In the future, we hope to get more servers.


Charity Engine - About Us

Charities - Research

Oh! I'd never heard of it before, but Matt Blumberg is part of their Tech Team, so I guess he would know.

FYI: A few weeks ago I found a pc here in The Netherlands that had the charityengine client installed without the user's knowledge. It is not impossible that some malware is responsible for this, the author of that malware would have more chance of winning prize money from charityengine. It was using a weird account name which I can't remember unfortunately.

The charityengine software is a clone of Boinc, but with their own icons.

By the way, this thread is getting really long and takes quite a bit of time to load. Maybe it's time for a new one?
ID: 77242 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 77243 - Posted: 3 Aug 2014, 8:34:03 UTC - in response to Message 77242.  

By the way, this thread is getting really long and takes quite a bit of time to load. Maybe it's time for a new one?


You can define the number of posts per thread you wish to load in the configuration of preferences on the R@h website. If a thread has more posts than you configure, there is a link you may use to see the remaining posts.
Rosetta Moderator: Mod.Sense
ID: 77243 · Rating: 0 · rate: Rate + / Rate - Report as offensive
AtHomer
Avatar

Send message
Joined: 26 Jan 10
Posts: 13
Credit: 7,180,613
RAC: 48
Message 77244 - Posted: 3 Aug 2014, 9:30:05 UTC - in response to Message 77243.  

By the way, this thread is getting really long and takes quite a bit of time to load. Maybe it's time for a new one?


You can define the number of posts per thread you wish to load in the configuration of preferences on the R@h website. If a thread has more posts than you configure, there is a link you may use to see the remaining posts.

I mostly read the message boards whilst nog being logged in. And then it always loads the entire thread in one page.

When I'm in a hurry, next time I'll log in first :-)
ID: 77244 · Rating: 0 · rate: Rate + / Rate - Report as offensive
[TA]Assimilator1
Avatar

Send message
Joined: 9 May 07
Posts: 7
Credit: 5,473,781
RAC: 162
Message 77246 - Posted: 3 Aug 2014, 10:42:02 UTC
Last modified: 3 Aug 2014, 10:51:30 UTC

Yea it doesn't make much sense to have 1 never ending thread for all different tech problems, the longer the thread gets the longer it takes to find out about new problems in it, especially if it's your 1st visit to it!

Yea you can look at posting dates but that's assuming the problem your looking at started at the same for you as others.

Oh btw all my uploads have gone so it seems to be sorted now :).

So are you project guys sure that all those new accounts are legit??
They look mighty dubious.......

Gallstone
Saying 'Damned' by itself makes no sense, you should of either said 'be damned' or 'damn you' ;)

Joking aside, yea the initial lack of feedback or homepage news about this problem was a poor show, but other than that I think you & some others throwing a hissy fit about this is pretty childish! Computers go wrong, internet problems happen, hardware can fail & hackers can screw things up, basically shit happens so deal with it! I just switch to another project when these sort of outages happen.
Although I will say it would be fair enough to get annoyed if you lost credit, did you? I'm guessing that Rosetta extended deadlines & so you didn't?........
Team AnandTech - SETI@H, Muon1 DPAD, F@H, MW@H, A@H, LHC@H, POGS, R@H, DHEP, CPDN

Main - Ryzen 5 3600, TR Ultima90, MSI B450, 32GB DDR4 3200, RX580 8GB, Seasonic Prime PX-550
2nd - i7 4930k @4.1GHz, TR Ultra120 E, 16GB DDR3 1866, HD7870 XT 3GB(DS
ID: 77246 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Sascha Becker

Send message
Joined: 1 Oct 05
Posts: 2
Credit: 4,318,835
RAC: 0
Message 77249 - Posted: 3 Aug 2014, 11:36:32 UTC - in response to Message 77248.  

Have a problem with the down / upload of Wu's for about 2 days. Currently, I have 13 Wu's done, but which are not uploaded either by my internet connection or the internet my parents. I have already tried two different routers. Times have issued terminates the killer virus or firewalls. But no up or download from Wu's. Can someone help me? The problem I have only with Rosetta @ home otherwise everything works. Sincerely oetker201 / Sascha Becker.


03.08.2014 13:33 The work units I could now upload.


Because of the problems with Rosetta @ home SETI @ home I had reactivated.
ID: 77249 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5770
Credit: 6,139,760
RAC: 0
Message 77253 - Posted: 3 Aug 2014, 15:06:47 UTC
Last modified: 3 Aug 2014, 15:13:05 UTC

So what is being done to "future" proof this project and prevent such overloads from happening?

Is there a way to allow new accounts to be added, but not crash the system like this?

What if anything can you guys do different that will prevent this from happening again? Is there some sort of automation sequence you can write or more hardware or modern hardware to install that could allow for such a big wave of new users to enter the system without crashing it?

Credits to me don't matter, just that my BOINC manager was getting clogged up with Rosie's problems and I have other projects that use my system that also fill the screen, so it was turning into a mess. Plus it seemed since the tasks could not be reported or uploaded BOINC manager did not know how to allocate resources or so it seemed.

Also, we have been asking for years now for someone to keep the main page up to date with info about problems or other news. Like now, the only main page post about this problem is some technical stuff by KEL. Nothing saying it has been solved. Though I guess you can get all that from here.

Communication to the outside non scientific world has always been a challenge for this project and I have suggested before that hiring or getting a volunteer from the communications department student pool would be a plus. They could be the projects PR/spokesperson. Also it seems internal communication is problem.
Yes one has a right to dump their phone and computer and return to the basics, but it would be nice if there was a backup person that knew about things like this charity organization and then could tell the others that a big clump of new users could be coming online.

Anyway..hope you guys learned something from this and will improve things for the future.

Happy Crunching...
ID: 77253 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Previous · 1 . . . 34 · 35 · 36 · 37 · 38 · 39 · 40 . . . 55 · Next

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home



©2026 University of Washington
https://www.bakerlab.org